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INTRODUCTION: 

This  24-month  effort  assisted  in  the  preparation  and  transition  of  DANA  (Defense 
Automated  Neurobehavioral  Assessment),  the  next-generation  neurocognitive  assessment 
tool  (NCAT),  into  operational  military  use,  as  well  as  prepared  DANA  for  use  in  CONUS 
clinics.  DANA  is  a  clinical  decision  support  tool  developed  for  and  funded  by  the 
Department  of  Defense  (DOD)  for  use  in  field  and  clinical  settings.  The  effort  was  organized 
around  two  foci  -  science  and  transition.  The  second  thrust,  transition,  included  obtaining 
FDA  clearance  for  DANA  (DOD  has  determined  it  is  a  medical  device),  and  positioning  DANA 
to  be  operationally  deployed  into  the  military. 


KEYWORDS:  Neurocognitive  Assessment  Tool,  DANA,  cognition,  concussion,  depression, 
PTSD,  transition,  database,  FDA  clearance,  normative  data 

ACCOMPLISHMENTS: 

What  were  the  major  goals  of  the  project? 

Science  Objectives 

Task  1:  IRB  approval  for  data  collection  for  three  studies 
Task  2:  Subject  Identification  for  the  three  studies 
Task  3:  Data  Collection  for  three  studies 
Task  4:  Data  Analysis  for  three  studies 
Task  5:  Reporting  for  three  studies 

Transition  Objectives 

Task  6:  Obtaining  FDA  Clearance 

Task  7:  DANA  Reliability  and  Validity  Publications 

What  was  accomplished  under  these  goals? 

Key  Research  Accomplishments 

•  Submission  of  a  510(k)  application  to  the  FDA  and  obtaining  clearance  for  DANA. 

•  Development  of  a  relational  database  to  store  our  datasets,  as  well  as  created  an 
automated  way  of  cleaning  our  datasets  to  prepare  them  for  data  analysis. 

•  Obtainment  of  normative  data  collection  on  a  military  population  in  order  use  this 
data  as  a  standard  reference  measure  among  military  populations. 

•  JHU  (depression  study):  preliminary  data  analysis  shows  that  DANA  provides  more 
sensitivity  than  the  MMSE  in  a  clinical  population  diagnosed  with  major  depressive 
disorder.  Results  of  this  study  are  forthcoming  as  data  analysis  is  currently 
underway.  (Preliminary  analysis  in  Appendices  0  and  P) 

•  The  VA  study  (PTSD  study)  showed  that  DANA  could  be  used  to  track  cognition  over 
time  in  a  home  based  telehealth  model.  (Preliminary  analysis  in  Appendix  Q) 

•  UW  (concussion  study):  preliminary  findings  show  that  the  DANA  Spatial  sub-test 
may  be  sensitive  to  concussion.  (Preliminary  analysis  in  Appendix  N) 

•  Numerous  manuscripts  have  been  published  showing  that  DANA  a  reliable, 
consistent  assessment  tool. 

•  DANA  is  being  evaluated  by  an  Integrated  Product  Team  (IPT)  as  a  step  in  the 
acquisition  process. 

Task  1:  IRB  approval  for  data  collection 

Once  the  revised  SOW  for  this  effort  was  approved,  we  moved  forward  in  obtaining  IRB 
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approval  for  each  study  proposed.  In  each  of  our  studies,  we  partnered  with  another 
organization,  and  as  the  data  was  collected  at  that  partner's  facility,  and  under  their 
administration,  each  host  was  required  to  submit  the  specific  protocol  for  that  study  to 
his/her  respective  IRB.  Once  the  study-specific  IRB  had  approved  the  protocol,  it  was 
submitted  to  MRMC  for  Army  IRB  approval.  To  date,  all  three  studies  obtained  approval  and 
are  in  the  data  analysis  stages. 

Task  2:  Subject  Identification  for  the  three  studies 

Following  guidance  from  the  Scientific  Advisory  Panel  members  and  as  indicated  in  the 
revised  Statement  of  Work  (approved  6/18/13),  we  identified  classes  of  subjects  for  the 
following  studies  proposed: 

1  Concussion  Study  (IRB  Protocol  #:  PRO  16061) 

We  collaborated  with  Dr.  Michael  McCrea  (Medical  College  of  Wisconsin)  and  Dr.  Alison 
Brooks  (University  of  Wisconsin).  The  purpose  of  this  study  was  to  compare  the 
assessment  of  cognitive  function  immediately  after  concussion  and  during  recovery 
from  moderate-severe  TBI  using  DANA  and  ImPACT.  250  student  athletes  in  football 
and  soccer  were  to  be  baselined  using  both  devices,  and  then  33  concussed  and  33 
matched  controls  were  to  be  assessed  for  concussion  in  a  direct  head  to  head 
comparison  over  the  season.  Measurements  included  pre-injury  baseline  assessment, 
acute  (<24  hours  post)  evaluation,  and  follow-up  assessments  8, 15  and  45  days  post¬ 
injury.  To  date,  250  athletes  have  been  baselined,  as  well  as  20  concussed  and  20 
matched  controls. 

2  Depression  Study  (Application  #:  NA_00082172) 

DANA  Rapid  was  used  to  assess  cognitive  functioning  in  patients  with  major  depressive 
disorder  undergoing  Electroconvulsive  Therapy.  We  believe  DANA  could  provide  a 
more  sensitive  assessment  of  cognitive  function  following  therapy  than  current 
assessment  tools.  Baseline  measures  were  recorded  prior  to  therapy,  then  again  at 
three  intervals  after  therapy.  An  N  of  20  ECT,  and  20  controls  was  anticipated.  Drs. 
Adam  Kaplin,  Steven  Woods  and  Kristen  Rahn,  Departments  of  Psychiatry  and 
Neurology,  Johns  Hopkins  University  School  of  Medicine,  led  the  study.  17  subjects 
were  baselined  in  the  ECT  treatment  group  and  16  in  the  non-ECT  group,  and  data 
collection  has  been  completed. 

3  PTSD  Study  (Project  #:  2013-03/JLS/PROMISE0007) 

The  primary  purpose  of  the  PTSD  study  conducted  by  our  partners,  Pacific  Health 
Research  and  Education  Institute  (PHREI),  was  to  provide  psychological  data  and  to 
analyze  the  longitudinal  data  in  support  of  the  transition  of  DANA  to  operational  use. 
Using  DANA,  clinicians  and  research  assistants  collected  longitudinal  data  from  Service 
Members  and  civilians  for  analysis  by  Dr.  Spira  and  his  team.  The  aggregate  data  will 
be  used  to  establish  and  confirm  referential  norms  and  recovery  patterns.  ATinc  is 
responsible  for  data  organization,  additional  analysis,  and  reporting.  48  subjects  have 
been  baselined,  all  in  the  experimental  treatment  group,  and  data  collection  has  been 
completed. 

4  Additional  studies  considered 
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We  considered  conducting  a  fatigue  study  at  WRAIR  as  well  as  a  daily  "ALERTNESS" 
study  at  NICoE,  but  the  length  of  time  needed  to  obtain  the  necessary  IRB  approval 
threatened  to  render  an  unacceptable  constraint  in  terms  of  data  collection  and 
analysis.  We  are  proceeding  with  seeking  approval,  and  depending  on  when,  and  if, 
obtained  will  use  IRAD  dollars  to  proceed  with  this  study.  As  of  the  final  report,  WRAIR 
has  not  received  IRB  approval  for  this  effort. 

While  we  were  unable  to  launch  a  fatigue  study  directly  with  WRAIR,  WRAIR  has 
pursued  separate  studies  with  DANA.  WRAIR  presented  a  poster  at  MHSRS  2015  on 
DANA  that  examined  the  relationship  between  neurocognitive  and  driving 
performance  during  50  hours  of  sleep  deprivation  in  subjects  receiving  caffeine  or 
placebo.  Performance  over  time  was  assessed  and  the  relationships  between 
performance  metrics  were  sought  in  an  effort  to  predict  operational  performance  using 
DANA.  (Appendix  I) 

Task  3:  Data  Collection 

Data  collection  has  been  completed  for  the  three  studies  described  below.  For  the 
concussion  study,  we  partnered  with  the  University  of  Wisconsin  and  have  baselines  for  82 
football  athletes.  For  the  PTSD  study,  we  partnered  with  the  VA  and  have  baselines  for  73 
subjects.  On  the  Depression  study  with  JHU,  we  have  17  patients  consented  for  the  ECT 
group  and  16  for  the  control  group. 

Normative  Data  Collection: 

A  healthy  population  of  active-duty  military,  aged  18-64,  provide  a  relevant  range  of 
subjects  by  which  a  normative  database,  representative  of  the  larger  population  of  active- 
duty  members  can  be  used  for  the  purpose  of  neurocognitive  test  comparison  when 
baseline  information  is  not  provided  or  cannot  be  easily  obtained. 

In  order  to  determine  how  many  subjects  per  age  bin  were  required  for  comparison  we  ran 
power  analysis  calculations  (Appendix  V).  It  was  determined  based  off  of  these  calculations 
that  50  subjects  per  age  bin  was  the  minimum  number  required  to  yield  significant  data. 

(Appendix  V). 

Normative  data  was  collected  under  the  direction  of  Dr.  Michael  Russell  at  Ft.  Hood.  We 
obtained  normative  data  for  824  subjects  (240  female,  and  584  male).  We  have  stored  and 
coded  the  norms  in  our  database  by  age  bins  ranging  from  18-64  years  old  as  well  as 
gender. 

Task  4:  Data  Analysis 

Significant  preparations  were  made  with  respect  to  performed  analyses.  These  include: 

•  Meeting  several  times  with  data  analysts  to  receive  statistical  advice  on  different 
ways  to  approach  longitudinal  correlations  and  data  analysis. 

•  Conducting  analysis  of  current  cross-sectional  and  longitudinal  DANA  data  to 
determine  sensitivity  and  specificity  for  power  estimates  of  DANA  studies  to  be 
conducted  as  part  of  the  RIF  protocols. 

•  Consulting  with  various  statisticians  to  reach  consensus  on  how  to  handle 
anomalous  data,  e.g.,  lapses,  anticipatory  responses,  outliers. 
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•  Extensive  discussions  with  statisticians  to  determine  the  optimal  sample  size  of 
treatment  and  control  groups.  With  some  of  the  more  complex  study  designs,  we 
made  a  deliberate  attempt  to  optimize  subject  size  so  as  to  minimize  unnecessary 
costs  and  arrive  at  the  most  efficient  design. 

•  Reviewing  and  selecting  RIF  longitudinal  protocols. 

•  Reviewing  past  DANA  pilot  studies  to  accurately  ascertain  inter  class  reliability 
(ICC)  and  reliable  change  indexes  (RCI). 

•  Developed  a  relational  database  to  house  all  study  data  and  created  an  automated 
cleaning  system  in  order  to  streamline  our  ability  to  quickly  begin  data  analysis. 

Study  Outcomes: 

1.  UW  Concussion  Study  (IRB  Protocol  #:  PRO  16061) 

Baseline  Testing  Overview: 

•  Baseline  testing  started  on  2/26/14  and  completed  3/3/14  for  spring  2014  football- 
82  athletes. 

•  Baseline  testing  started  on  6/4/2014  and  completed  by  9/22/14  for  2014- 
2015  season  for  football,  soccer,  basketball,  ice  hockey  -  228  athletes. 

•  August  2014-7  injured  athletes  started  post-injury  testing. 

•  September  2014-2  injured  athletes  started  post-injury  testing;  7  controls  started 
testing;  4  injured  athletes  completed  day  45  testing. 

•  October  2014  -  3  injured  athletes  started  post-injury  testing;  5  controls  started 
testing;  3  injured  athletes  completed  day  45  testing;  5  controls  completed  day  45 
testing. 

•  November  2014-2  injured  athletes  started  post-injury  testing;  1  control  started 
testing;  4  injured  athletes  completed  day  45  testing;  4  controls  completed  day  45 
testing. 

•  December  2014  -  2  injured  athletes 

•  January  2015  -  2  injured  athletes 

•  March  2015  -  7  injured  athletes 

•  April  2015-3  injured  athletes 

•  As  of  June  8,  2015,  all  data  collection  and  entry  is  complete  with:  232  DANA 
baselines.  There  were  28  concussed  athletes  -  (14  football,  4  women's  soccer,  4 
men's  soccer,  2  men's  basketball,  3  women's  hockey  and  1  men's  hockey).  All  28 
concussed  athletes  and  controls  completed  post-injury  testing. 

•  Data  Analysis,  Interpretation,  Reporting  and  Dissemination  are  now  currently  in 
progress.  Manuscript  preparation  will  be  on  going  as  data  is  analyzed. 

•  Internal  preliminary  data  analysis  was  completed  comparing  healthy  controls  with 
concussed  (Report  is  in  Appendix  N). 

2.  IHU  Depression  Study  (Application  #:  NA  00082172) 

•  Completed  study  recruitment;  final  study  numbers  are  N=16  for  controls,  and  N=17 
for  ECT  patients.  Data  analysis  is  now  underway. 

•  Presented  oral  presentation  at  a  Public  Health  Conference  in  Bangkok,  Thailand  on 
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the  "Longitudinal  evaluation  using  the  recently  developed  Defense  Automated 
Neurobehavioral  Assessment  (DANA)  tool  of  the  cognitive  impact  of 
Electroconvulsive  Therapy  (ECT)  in  the  treatment  of  Major  Depressive  Disorder 
(MDD)" 

•  Presented  poster  at  the  2014  Military  Health  Systems  Research  Symposium 
(MHSRS).  (Appendix  G) 

•  Poster  presented  to  Johns  Hopkins  University  School  of  Medicine  Psychiatry 
Research  Symposium. 

•  Presented  abstract  on  October  20,  2014  to  the  Health  Information  Technology 
Conference  in  Baltimore,  MD.  (Appendix  K) 

•  Preparing  manuscript  on  initial  findings  of  correlation  between  DANA  and  the  Mini 
Mental  Status  Exam  (MMSE).  [Preliminary  analysis  in  Appendix  0) 

3.  VA  PTSD  Study  fProiect  #:  2013-03/ILS/PRQMISE00071 

•  Obtained  normative  data  using  DANA  on  73  older  and  female  veterans  in 
August/September  2014. 

•  Longitudinal  data  for  48  completed  participants  received  by  HBTMH  research  team 
and  extracted  from  devices  for  analysis.  28  participants  completed  3  DANA 
assessments;  9  participants  dropped  out  after  2  assessments,  and  11  only 
completed  one  assessment. 

•  Cleaned,  coded,  and  formatted  data  for  input  into  SPSS  for  data  analysis. 

•  Analysis  of  data  and  manuscript  preparation  is  underway.  [Interim  analysis  in 
Appendix  Q,  slides  "Interim  Analysis"  through  "Next  Steps"). 

•  Ran  data  analysis  measures  on  DANA  for  test  reliability  (test-retest,  split  half,  alpha 
correlations),  internal  validity  (factor  analysis),  as  well  as  sensitivity  and  specificity 
(ROC  and  logistic  regression). 

Additional  Completed  Data  Analysis: 

Of  the  data  analysis  below,  no  human  subjects  research  was  conducted  under  this  RIF 

project.  The  data  overview  below  was  implemented  under  other  funding  during  the  period 

of  the  RIF  and  is  included  as  a  reference  point  for  further  validation  of  DANA. 

Mixed  Martial  Arts  (MMA)  Data: 

•  Data  for  19  MMA  athletes  were  collected.  The  goal  of  this  data  collection  was  to 
compare  subjects  with  multiple  concussions  (MMA)  vs.  those  with  acute  concussion. 
An  exploratory  analysis  was  completed.  [Report  in  Appendix  R). 

Air  Force  Academy  Football  Data: 

•  Data  for  153  healthy  Football  Athletes  were  collected  along  with  6  concussed.  The 
goal  of  this  data  collection  was  to  examine  what  the  patterns  of  acute  concussion 
data  were  over  time.  [Report  in  Appendix  S). 

In-Theater  Afghanistan  Data  Analysis  (DANA  vs.  ANAM): 

•  An  analysis  was  completed  for  the  ANAM  test  battery  against  DANA.  The  goal  of  the 
analysis  was  to  understand  how  DANA  measures  differ  from  ANAM  measures  on 
similar  sub-tests.  [Report  in  Appendix  T). 

Altitude  Data  Analysis: 
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•  Study  comparing  cognition  at  different  altitudes  using  DANA  ( Analysis  in  Appendix 
U).  For  additional  details  please  see  publications  titled  “AltitudeOmics:  the 
integrative  physiology  of  human  acclimatization  to  hypobaric  hypoxia  and  its 
retention  upon  renascent "  (Appendix  B)  and  “AltitudeOmics:  Decreased  reaction 
time  after  high  altitude  cognitive  testing  is  a  sensitive  metric  of  hypoxic  impairment ". 
(Appendix  C) 

Power  Analysis  Calculations  for  Normative  Data: 

•  Normative  data  collection  power  analysis  done  to  identify  number  of  subjects 
needed  to  collect  for  reference  normative  database.  ( Appendix  V) 

Task  5:  Reporting 

DANA  includes  an  on-screen  reporting  feature  for  users  as  well  as  in-depth  exporting  report 
functionality  for  researchers  that  includes  detailed  trial-by-trial  information  for  further 
statistical  analysis.  Both  on-screen  and  detailed  report  formats  are  generally  specific  to  the 
user  community's  need  (e.g.  clinical,  military,  etc.).  For  each  community,  customization  may 
be  needed  to  provide  the  most  optimal  information  required  for  their  specific  use.  The 
current  DANA  reporting  function  is  generalized  since  an  end  user  has  not  been  finalized; 
our  general  reports  are  based  off  interviews  with  a  number  of  clinicians  as  well  as  extensive 
feedback  received  from  our  partners  who  are  currently  using  DANA.  Once  we  have 
identified  our  target  transition  partner  and  are  ready  to  facilitate  the  final  format,  we  will 
ensure  expert  feedback  and  assessment  of  the  ease  of  understanding  the  reports  that  are 
generated. 

Also  in  this  reporting  period,  we  worked  with  the  Federal  Interagency  Traumatic  Brain 
Injury  Research  (FITBIR)  informatics  system  to  get  our  data  in  the  correct  format  for 
upload.  FITBIR  was  designed  by  the  NIH  and  Department  of  Defense  to  facilitate  research 
and  create  an  environment  for  collaboration. 

Because  DANA  RIF  is  a  DOD-funded  grant,  we  were  to  upload  the  data  collected  from  the 
Concussion  Study  (UW)  to  the  FITBIR  database.  We  have  completed  the  creation  of  our 
FITBIR  account,  have  worked  with  them  on  drafting  the  structure  of  our  data  template,  and 
have  completed  upload  of  the  data  from  the  UW  study. 

Database  Outcomes: 

The  AnthroTronix  in-house  database  called,  "research"  houses  all  data  directly  and 
indirectly  relating  to  the  DANA  project.  It  is  a  MySQL  relational  database  that  provides  the 
capability  to  link  together  various  types  of  data  contained  within  it.  "Research"  holds  both 
cognitive  and  psychological  test  information,  by  summary  and  by  trial.  (Database  diagram 
in  Appendix  M) 

Additionally,  in  order  to  streamline  data  analysis,  we  developed  a  way  automatically  parse 
and  clean  data.  We  first  developed  routines  by  subtest  that  would  take  large  files  of  data 
and  separate  them  into  information-by-subject,  compute  the  necessary  and  widely  used 
summary  statistics,  and  also  flag  subjects  who  performed  sub-par  due  to  issues  such  as 
malingering;  this  automation  process  rendered  files  that  were  readied  for  easy  export  into 
the  database.  When  in  the  database,  it  can  be  accessed  easily,  and  each  type  of  metric  can  be 
related  to  other  types  of  metrics. 

A  relational  database  is  object-constructed  -  categorically,  not  hierarchically.  The  contents 
of  a  relational  database  are  as  follows: 
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Tables  "Relations"  that  metaphorically  represent  objects  contained  therein  by  tabular 
format.  A  table  is  an  implementation  of  a  "base  relation/'  signifying  that  data  is  stored  for 
access. 

Columns  Fields  of  a  record,  or  in  database  vocabulary,  "attributes." 

Rows  Data  value  records  or  "tuples." 

Views  Other  types  of  relations  that  represent  objects  contained  therein  by  tabular  format, 
but  a  view  is  an  implementation  of  a  "derived  relation,"  signifying  that  data  is  not  stored  for 
access,  but  rather  acts  as  a  single  relation  resulting  from  enactment  of  a  pre-constructed 
query;  essentially,  a  view  provides  in  one  place  information  gathered  from  many  tables, 
without  utilizing  the  permanent  storage  space  of  the  database.  Every  view  in  "research"  is 
designated  with  an  ending  letter,  "q". 

The  MySQL  Workbench  application  allows  one  to  visually  structure  a  database  or  schema, 
with  the  use  of  an  enhanced  entity-relationship  (EER)  diagram.  (DANA  EER  in  Appendix  M) 

Task  6:  Transition  DANA  for  Deployment 

Task6A:  Prepare  and  File  FDA  510(k);  Task  6B:  Implement  Quality  System 
processes  per  FDA  21  CFR  Part  820;  Task  6C:  Software  Documentation  for  510(k) 
application;  Task  6D:  Literature  Review  of  pre  1976  Test  Artifacts;  Task  6E: 
Additional  Documentation  in  Support  of  510(k)  Application 
In  order  for  DANA  to  transition  for  use  by  the  DOD,  it  needed  to  be  reviewed  and  cleared  by 
the  FDA  as  a  mobile  medical  device.  To  that  end,  we  submitted  a  513(g)  application  to  the 
FDA  requesting  determination  as  to  its  classification,  i.e.,  Class  I,  II  or  III  in  April  2013.  We 
received  feedback  from  the  FDA  regarding  our  513(g)  application,  and  the  FDA's  response 
was  that  DANA  was  to  be  considered  a  Class  III  device  subject  to  a  510(k)  application.  Due 
to  this  classification  we  moved  ahead  with  working  on  a  510(k)  application,  which  was 
completed  and  submitted  in  July  2014.  The  FDA  determined  that  DANA  was  an  unclassified 
device,  and  we  received  FDA  clearance  on  October  15,  2014.  (Clearance  Letter  in  Appendix 
X) 

During  the  process  of  submitting  a  510(k)  to  obtain  FDA  clearance,  ATinc  worked  with  the 
FDA  to  provide  sufficient  information  and  documentation  on  DANA  and  to  make  sure  DANA 
met  the  necessary  criteria  for  clearance.  There  were  several  steps  taken  as  part  of  this 
process.  We  found  a  predicate  device — the  QB  Test — that  best  matched  the  intended  use  of 
DANA  and  could  be  compared  to  as  part  of  the  510(k).  Both  DANA  and  the  QB  Test  are 
Mobile  Based  Task  Performance  Recorders,  DANA  because  it  measures  reaction  time  on  a 
mobile  platform. 

Furthermore,  we  restructured  our  software  quality  process  to  match  the  guidelines  set  out 
by  the  FDA.  We  also  discussed  how  to  classify  DANA  with  the  FDA;  it  was  originally 
considered  a  Class  III,  but  we  proved  that  high  of  a  class  not  to  be  necessary  (of  low  risk, 
etc.),  and  DANA  is  currently  unclassified.  We  analyzed  the  safety  risks  and  hazards  of  DANA 
and  made  sure  they  were  within  acceptable  limits.  This  was  a  process  of  testing  and 
evaluating  the  software,  determining  its  level  of  concern  (minor),  and  documenting  all 
testing  and  results  as  well  as  corrective  measures  and  design  elements  that  mitigate  or 
eliminate  said  risks  and  hazards.  DANA  was  found  to  have  few  hazards  and  risks  associated 
with  it.  In  addition,  we  created  labeling  stating  the  indications  of  use  and  providing 
instructions  elucidating  those  indications  that  met  FDA  requirements  for  labeling. 
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Outcomes: 

Obtaining  clearance  attests  that  DANA  is  a  safe  and  effective  device.  As  part  of  our  FDA 
510(k)  application,  we  developed  extensive  supporting  documentation;  the  documents  are 
outlined  below.  DANA  receiving  FDA  clearance  (Appendix  X)  confirms  the  value  of  DANA  as 
a  neurocognitive  assessment  tool  for  use  by  medical  personnel. 

510(k)  premarket  notification — demonstrates  that  the  device  to  be  marketed  is  at  least  as 
safe  and  effective,  that  is,  substantially  equivalent,  to  a  legally  marketed  device.  Ours 
included  the  following  sections: 

•  Medical  Device  User  Fee  Cover  Sheet  (Form  FDA  3601) 

•  CDRH  Premarket  Review  Submission  Cover  Sheet 

•  510(k)  Cover  Letter 

•  Indications  for  Use  Statement 

•  510(k)  Summary  or  510(k)  Statement — brief  summary  of  the  device  included  in  the 
510(k)  and  the  supporting  information. 

•  Truthful  and  Accuracy  Statement — statement  certifying  that  all  information 
submitted  in  the  510(k)  is  truthful  and  accurate  and  that  no  material  fact  has  been 
omitted. 

•  Executive  Summary — concise  description  of  the  device,  including  the  indications  for 
use  and  technology;  device  comparison  table;  concise  summary  for  any  performance 
testing  in  the  submission. 

•  Device  Description — performance  specifications  and  brief  description  of  the  device 
design  requirements. 

•  Substantial  Equivalence  Discussion — identify  the  predicate  device;  provide  a 
detailed  comparison  with  our  device  in  terms  of  indications  for  use,  technology  and 
performance  specifications.  Our  predicate  device  was  the  QB  Test. 

•  Software — appropriate  software  documentation;  identification  of  the  "level  of 
concern"  and  documentation  thereof.  A  Safety  Risk  Analysis  document  was 
developed  that  provides  a  comprehensive  assessment  of  potential  hazards  that  may 
occur  in  the  use  of  DANA.  Contains  descriptions  of  Risk  Rating  Criteria,  Modes  of 
Control  and  Document  References.  Also  includes  the  Safety  Risk  Analysis  Table  and 
Safety  Requirements  by  mode  of  control.  This  is  proprietary  information  and  thus, 
not  included  in  this  report. 

•  Indications  for  Use 

Identification  and  description  of  the  specific  indications  for  use  statement  for  the 
device(s).  It  identifies  the  target  population  in  a  significant  portion  of  which 
sufficient  valid  scientific  evidence  has  demonstrated  that  the  device  as  labeled  will 
provide  clinically  significant  results  and  at  the  same  time  does  not  present  an 
unreasonable  risk  of  illness  or  injury  associated  with  the  use  of  the  device. 

•  Software  Documentation 

The  software  documentation  for  the  FDA  clearance  is  proprietary  and  therefore  not 
included  in  this  report. 

•  Standard  Neuropsychological  Questionnaires 

Full  text  and  scoring  info  for  neuropsychological  questionnaires  that  are  included  in 
DANA  (e.g.  PHQ-8). 

•  Proposed  Labeling 

Labeling"  includes  the  device  label,  instructions  for  use,  and  any  patient  labeling. 
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The  labeling  for  the  FDA  clearance  is  proprietary  and  therefore  not  included  in  this 
report. 

Other  Transition  Activities 

A  number  of  paths  exist  for  this  transition,  and  our  first  success  to  date  is  with  the 
Neurocognitive  Assessment,  Rehabilitation  and  Reintegration  Product  Line,  UNITED 
STATES  ARMY  MEDICAL  COMMAND.  Dr.  Michael  Russell,  the  ANAM  Program  Director,  and 
his  successor,  Dr.  Watson,  have  commissioned  DANA  for  a  norming  study  they  are 
conducting  for  the  US  Army.  To  date,  data  from  824  subjects  has  been  collected.  We  now 
have  norms  for  the  age  bins  of  18-19,  20-24,  25-29,  30-34,  35-44,  45-54,  and  55-64. 

In  addition,  we  have  offered  to  work  closely  with  an  integrated  product  team  (IPT),  headed 
by  Dr.  Valerie  Trabosh  and  LTC  Atchison  at  MRMC,  specifically  set  up  for  this  purpose,  and 
we  await  the  IPT's  guidance  and  direction.  As  of  this  final  report  LTC  Atchison  had 
introduced  ATinc  to  the  US  Army  Rapid  Equipping  Force,  and  we  were  ready  to  submit 
DANA  to  an  interoperability  network  study,  but  REF  delayed  that  opportunity. 

Similarly,  we  discussed  the  development  of  a  “performance  app",  based  on  a  subset  of  DANA 
tests,  to  be  distributed  through  DARPA's  Transformative  Apps  program,  but  further 
progress  on  that  front  will  depend  on  identifying  a  customer  and  a  set  of  requirements. 

Also  with  DARPA's  help,  we  submitted  a  DANA  Android  application  package  file  (APK)  for 
them  (DARPA)  to  review.  It  is  necessary  for  DARPA  to  “clear"  or  “vet"  DANA  software  for 
hosting  on  military  computer  systems.  Since  the  DARPA  secure  kernel  was  developed  by  a 
contractor,  Invincea,  the  operating  system  is  proprietary,  and  we  have  invited  the  program 
manager  to  install  the  DARPA  kernel  on  our  devices  so  that  we  can  be  assured  that  device 
timing  is  not  negatively  impacted.  As  of  this  final  report  there  are  no  updates  from  DARPA. 

We  completed  a  research  and  interview  campaign  targeting  opportunities  within  the  U.S. 
Department  of  Defense  and  the  U.S.  Intelligence  Community  (including  DHS,  State)  with 
need  and  budget  for  Traumatic  Brain  Injury  assessment  and  monitoring.  Similarly,  we 
compiled  research  and  inquiries  into  an  unclassified  opportunity  report  that  will  help  with 
DANA  commercialization  and  new  product  market  entry. 

Task  7:  DANA  Reliability  and  Validity  Publications 

To  further  verify  the  reliability  and  validity  of  DANA,  we  developed  a  trade  study  reflecting 
the  work  done  to  address  questions  regarding  the  sensitivity,  specificity  and  discriminative 
validity  of  DANA. 

During  the  course  of  this  RIF  contract,  we  have  been  able  to  publish  our  findings  in  a  variety 
of  journal  publications.  The  manuscripts  resulting  from  our  research  in  the  past  2  years  are 
listed  under  the  “Products"  section  of  this  report  starting  on  page  14. 

Conclusions 

All  studies  have  completed  data  collection.  Data  analysis  and  manuscript  preparation  are 
currently  underway.  Ongoing  analysis  shows  promise  regarding  DANA's  applicability,  in 
part  because  several  key  areas  important  to  DANA's  usability  in  CONUS,  including 
depression,  concussion,  and  clinical  treatment  populations,  are  being  looked  at.  We  will  be 
presenting  a  poster  on  ongoing  validation  of  DANA  with  Fort  Hood  norms  at  this  year's 
Military  Health  Systems  Research  Symposium. 

Partnering  with  other  institutions  has  aided  us  in  fulfilling  both  the  transition  and  science 
efforts.  DANA  data  was  successfully  integrated  with  FITBIR,  and  we  submitted  the  DANA 
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Android  application  package  file  (APK)  to  DARPA  for  vetting.  Our  studies  were  conducted  in 
collaboration  with  several  prestigious  organizations  whose  interest  in  DANA  demonstrated 
its  value,  and  the  data  collected  by  will  help  prepare  DANA  for  CONUS  clinical  use. 

Under  this  Rapid  Innovation  Fund  Contract,  ATinc  received  significant  feedback  and  input 
from  Dallas  Hack,  COL  (Rtd)  USA,  who  recently  retired  the  Brain  Health  Program 
Coordinator  in  the  Office  of  the  Principal  Assistant  for  Research  &  Technology,  US  Army 
Medical  Research  and  Material  Command  (MRMC].  ATinc  also  briefed  DANA  to  the 
Integrated  Product  Committee  within  MRMC  that  has  responsibility  for  this  area.  ATinc  has 
also  briefed  other  key  stakeholders,  including  the  following  individuals,  as  well  as  others: 

•  John  E.  Meyers,  Psy.D.;  ANAM  Program  Director/Chief  Neurocognitive  Assessment 
Branch  Rehabilitation  and  Reintegration  Division,  HQDA,  Office  of  the  Surgeon 
General  of  the  Army 

•  COL  Sidney  Hinds,  MD;  National  Director,  Defense  and  Veteran's  Injury  Brain  Center 
(DVBIC], 

•  Ms.  Kathy  Helmick;  Deputy  Director,  DVBIC 

•  LTC  Eugene  Kim,  MD;  Command  Psychiatrist,  United  States  Army  Special  Operations 
Command  (USASOC] 

•  CAPT  Jack  Tsao,  MD,  USN:  Director,  Traumatic  Brain  Injury  Programs,  Wounded,  Ill 
&  Injured  Directorate,  US  Navy  Bureau  of  Medicine  and  Surgery. 

At  the  2015  Military  Health  Systems  Research  Symposium  (MHSRS],  DANA  was  cited  by 
RADM  Bruce  Doll,  USN,  who  is  dual-hatted  as  the  Director,  Research,  Development  and 
Acquisition,  Defense  Health  Agency  and  as  Deputy  Commander,  US  Army  MRMC,  and  by  Dr. 
David  Smith,  the  Deputy  Assistant  Secretary  of  Defense  for  Force  Health  Protection  and 
Readiness,  as  a  key  success .  In  addition,  representatives  from  AnthroTronix  met  with  MAJ 
Walter  Carr,  USA  and  Dr.  John  Myers  to  discuss  possible  transition  opportunities. 

What  opportunities  for  training  and  professional  development  has  the  project 
provided? 

Nothing  to  Report. 

How  were  the  results  disseminated  to  communities  of  interest? 

Related  to  the  results  of  this  effort,  we  have  published  5  papers  and  plan  to  publish  several 
more.  DANA  was  utilized  in  5  posters  that  were  presented  at  MHSRS,  in  another  for  the  John 
Hopkins  University  School  of  Medicine  Psychiatry  Research  Symposium,  and  an  abstract 
that  was  presented  to  the  Health  Information  Technology  Conference  in  Baltimore,  MD. 
AnthroTronix  has  also  reported  quarterly  to  the  MRMC  and  briefed  DANA  to  the  IPT  within 
the  MRMC  as  well  as  to  other  relevant  organizations  and  persons  (see  Conclusions,  p.  13). 
What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 
Nothing  to  Report  as  the  project  has  ended.  However,  we  plan  to  publish  results  as  reports 
are  finalized. 

IMPACT: 

What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

DANA  has  great  potential  to  provide  a  better  way  for  health  practitioners  and  cognitive 
researchers  to  evaluate  cognition  and  to  track  it  over  time.  The  JHU  preliminary  data 
analysis  shows  that  DANA  provides  more  cognitive  sensitivity  than  the  MMSE  in  a  clinical 
population  diagnosed  with  major  depressive  disorder  (See  "Results" slide  through 
"Sample  Patient  #2"  in  Appendix  L).  Our  PTSD  study  showed  that  DANA  could  be  used  to 
track  cognition  over  time  in  a  home  based  telehealth  model,  and  UW  preliminary  findings 
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show  that  the  DANA  Spatial  sub-test  may  be  sensitive  to  concussion  (See  tables  "Baseline 
(controls)  vs.  45  Day(concussed)  meanThru ",  "Baseline  (controls)  vs.  15  day 
(concussed)  meanThru",  ",  "Baseline  (controls)  vs.  15  day  (concussed)  meanRT ", 
"Baseline  (controls)  vs.  24  hour  (concussed)",",  and  "Baseline  (controls)  vs.  8  Day 
(concussed)  percent  Correct"  in  Appendix  N). 

What  was  the  impact  on  other  disciplines? 

Nothing  to  Report. 

What  was  the  impact  on  technology  transfer? 

One  of  the  objectives  of  this  project  was  to  transition  DANA  to  the  military.  We  have  made 
great  strides  in  paving  the  way  for  DANA  to  be  transitioned  to  DoD  as  per  the  activities 
listed  in  Task  6  and  other  transition  activities.  In  addition,  there  is  great  interest  in  the 
clinical  health  market  for  DANA  based  on  findings  from  the  JHU  study.  AnthroTronix  will 
continue  to  pursue  transfer  activities  beyond  the  military. 

What  was  the  impact  on  society  beyond  science  and  technology? 

DANA  is  the  first  and  currently  the  only  NCAT  to  have  received  FDA  clearance.  DANA's 
clearance  is  a  significant  step  forward  for  healthcare  providers'  ability  to  treat  patients, 
because  there  are  very  few  tools  that  clinicians  and  other  health  providers  can  use  to  track 
changes  or  find  impairments  in  cognitive  functioning  and  no  others  that  are  FDA-cleared. 
With  clearance  and  the  scientific  development  under  this  effort,  DANA's  ability  to  be  used  in 
acute  and  non-acute  settings  to  identify  neurocognitive  issues,  isolate  symptoms,  and 
inform  the  proper  treatment  has  expanded. 

CHANGES/PROBLEMS: 

Changes  in  approach  and  reasons  for  change 

While  the  original  proposal  called  for  a  scientific  thrust  that  would  track  a  deployed 
battalion  OCONUS  longitudinally  over  time,  MRMC  leadership  (Colonels  Castro  and  Hack) 
agreed  that  the  statement  of  work  be  modified  to  concentrate  on  CONUS-based  studies  such 
as  testing  DANA  in  clinical  drug,  fatigue/alertness,  concussion  and/or  depression  protocols. 
Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 
A  delay  in  data  analysis  will  require  additional  publications  to  be  completed  this  coming 
year.  AnthroTronix  is  committed  to  publishing  all  results  from  these  studies  using  IRAD 
funds. 

Changes  that  had  a  significant  impact  on  expenditures 

Nothing  to  Report. 

Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animals,  biohazards, 
and/or  select  agents 

Nothing  to  Report. 

Significant  changes  in  use  or  care  of  human  subjects 

Nothing  to  Report. 

Significant  changes  in  use  or  care  of  vertebrate  animals  -N/A 
Significant  changes  in  use  of  biohazards  and/or  select  agents  -N/A 

PRODUCTS: 

Publications,  conference  papers,  and  presentations,  journal  publications 
2013: 
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Lathan,  Corinna,  etal.  "Defense  Automated  Neurobehavioral  Assessment  (DANA) - 
psychometric  properties  of  a  new  field-deployable  neurocognitive  assessment  tool." 
Military  medicine  1 78.4  (2013):  365-371. 

Published:  Yes.  Acknowledgement  of  federal  support:  Yes 

Describes  the  psychometric  properties  of  DANA,  based  on  the  assessment  of  224  service 
members  under  various  environmental  stresses:  extreme  cold,  humidity,  altitude,  dry  heat, 
and  unstable  platform  conditions.  DANA  results  were  compared  to  data  previously 
published  from  the  currently  used  Automated  Neuropsychological  Assessment  Metrics 
(ANAM)  tool.  DANA  performed  well  in  all  five  environments;  test-retest  reliability  for 
subtests  were  within  acceptable  ranges;  and  stability  was  consistent  with  ANAM's. 

2014: 

Subudhi,  Andrew  W.,  et  al.  "AltitudeOmics:  the  integrative  physiology  of  human 
acclimatization  to  hypobaric  hypoxia  and  its  retention  upon  reascent." PloS  one9.3 
(2014):  e92191. 

Published:  Yes.  Acknowledgement  of  federal  support:  No 

Subjects  were  tested  near  sea  level  then  at  different  altitudes.  They  were  tested  at  an 
altitude  of  5260m,  and  then  retested  at  that  altitude  after  being  acclimatized  to  a  lower 
altitude.  7  of  the  volunteers  stayed  at  a  lower  altitude  for  21  days,  while  14  stayed  for  7 
days.  Afterwards,  the  subjects  were  tested  again  at  the  high  altitude. 

Five  of  the  DANA  tests  showed  significantly  worse  scores  upon  arrival  at  5260m  and,  for  the 
subjects  that  reascended  after  seven  days,  two  tests  showed  retention  of  acclimatization. 

Roach,  Emma  B.,  et  al.  "AltitudeOmics:  Decreased  reaction  time  after  high  altitude 
cognitive  testing  is  a  sensitive  metric  of  hypoxic  impairment."NeuroReport  25.11 
(2014):  814. 

Published:  Yes.  Acknowledgement  of  federal  support:  Yes. 

DANA  was  used  to  examine  healthy  volunteers  at  sea  level  and  in  hypoxic  environments.  It 
was  administered  three  times:  at  sea  level,  when  the  subjects  arrived  at  the  destination 
altitude  of  5260m,  and  after  16  days  of  acclimatization.  There  was  a  significant  decrease  in 
performance  between  sea  level  and  the  other  tests.  The  SRT  change  score  was  measured 
(difference  between  two  reaction  time  scores  which  reflects  the  mental  effort  spent).  This 
score  provides  a  more  reliable  indication  of  hypoxia-induced  cognitive  impairment  than 
typical  baseline  procedures. 

Spira,  James  L.,  et  al.  "The  Impact  of  Multiple  Concussions  on  Emotional  Distress,  Post- 
Concussive  Symptoms,  and  Neurocognitive  Functioning  in  Active  Duty  United  States 
Marines  Independent  of  Combat  Exposure  or  Emotional  Distress.  "Journal  of 
neurotrauma  31.22  (2014):  1823-1834. 

Published:  Yes.  Acknowledgement  of  federal  support:  Yes. 

Examines  whether  concussions  had  a  sustained  impact  on  emotional  and  cognitive 
functioning  of  US  Marines.  646  Marines  were  tested  using  the  Standard  battery. 

Findings  indicated  that  recent  concussions  were  associated  with  more  distress  and  that 
having  three  or  more  lifetime  concussions  was  linked  with  the  worst  scores  for  depression, 
PTSD,  anger,  post-concussive  symptoms,  and  the  worst  performances  on  simple  cognitive 
tasks.  DANA  was  sensitive  to  factors,  showing  that  it  has  promise  as  a  tool  for  detecting 
changes  in  cognitive  functioning. 

2015: 
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Russo ,  C.  R.,  and  C.  E.  Lathan.  "An  Evaluation  of  the  Consistency  and  Reliability  of  the 
Defense  Automated  Neurocognitive  Assessment  Tool."  Applied  Psychological 
Measurement  (2015):  0146621615577361 

Published:  Yes.  Acknowledgement  of  federal  support:  Yes. 

Examines  Rapid  test  battery  consistency  in  a  US  Air  Force  Academy  football  player 
population,  who  were  tested  before,  during  and  after  the  football  season.  The  study 
evaluates  the  stability  of  the  test  and  describes  an  approach  to  determine  reliability  in 
terms  of  a  “minimum  difference"  threshold  for  the  detection  of  significant  change.  Study 
results  indicated  that  the  test  results  are  consistent  as  long  as  the  re-test  session  is 
administered  within  six  months  of  the  first  session. 

Books  or  other  non-periodical,  one-time  publications. 

Nothing  to  report. 

Other  publications,  conference  papers,  and  presentations. 

2014 

Eonta,  S.E.,  Paech,  G.M.,  Banks,  S.,  Johnson,  K.,  Della  Vedova,  C.,  Kamimori,  G.H.  (2014, 
August).  Impact  of  Caffeine  on  Neurocognitive  Performance  During  Sleep  Deprivations  Using 
the  Defense  Automated  Neurobehavioral  Assessment  (DANA).  Poster  Presented  at  the 
Military  Health  System  Research  Symposium,  Ft.  Lauderdale,  FL. 

Woods,  S.R.,  Lathan,  C.,  Susukida,  R.,  Riehm,  A.,  Askew,  H.,  Rahn,  K.,  Kaplin,  A.  (2014, 
August).  Longitudinal  Evaluation  Using  the  Recently  Developed  Defense  Automated 
Neurobehavioral  Assessment  (DANA)  Tool  of  the  Cognitive  Impact  of  Electroconvulsive 
Therapy  (ECT)  in  the  Treatment  of  Major  Depressive  Disorder  (MDD).  Poster  presented  at  the 
Military  Health  System  Research  Symposium,  Ft.  Lauderdale,  FL. 

Spira,  J.L.,  King,  L.A.,  Lathan,  C.,  Tsao,  J.  (2014,  August)  Development  of  an  Objective , 
Functional  Biomarker  of  Invisible  Wounds  of  War  for  Use  by  Providers.  Poster  Presented  at 
the  Military  Health  System  Research  Symposium,  Ft.  Lauderdale,  FL. 

2015 

Kamimori,  G.H.,  LaValle,  C.,  Paech,  G.,  Johnson,  K.,  Banks,  S.,  Dark,  H.,  Eonta,  S.  (2015, 
August).  Performance  on  the  Defense  Automated  Neurobehavioral  Assessment  Battery 
(DANA)  and  Simulated  Driving  Performance  over  50h  of  Continuous  Wakefulness:  Correlation 
between  Neurocognitive  and  Operational  Performance  (#2003).  Poster  Presented  at  the 
Military  Health  System  Research  Symposium,  Ft.  Lauderdale,  FL. 

Lathan,  C.,  Wallace,  R.,  Shewbridge,  R.  (2015,  August).  Differences  in  Cognitive  Efficiency  In 
Active  Duty  Military  -  Healthy  vs.  Those  with  Sleep  Disturbance ,  Depression ,  and/or  PTSD 
Using  the  DANA  Automated  Neurocognitive  Assessment  (DANA)  Tool.  Poster  Presented  at  the 
Military  Health  System  Research  Symposium,  Ft.  Lauderdale,  FL. 

Website(s)  or  other  Internet  site(s) 

Nothing  to  Report. 

Technologies  or  techniques 

Nothing  to  Report. 

Inventions,  patent  applications,  and/or  licenses 
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•  Provisional  patent  application:  61/597,068,  Performance  Assessment  Tool 
02/09/2012 

•  Non-provisional  patent  application:  US  13/764,162,  Performance  Assessment  Tool 
02/09/2012 

Other  Products 

Nothing  to  Report. 


PARTICIPANTS  &  OTHER  COLLABORATING  ORGANIZATIONS 
What  individuals  have  worked  on  the  project? _ 


Name: 

Dr.  Corinna  Lathan 

Project  Role: 

Principal  Investigator 

Nearest  person  month  worked: 

6 

Contribution  to  Project: 

Dr.  Lathan  interfaced  with  Drs.  Spira  and  Bleiberg  during  data 
management  and  reporting  and  supported  the  ATinc  team  of 
managers  and  engineers  throughout  the  project  during  technical 
development. 

Name: 

Akhila  Ananthram 

Project  Role: 

Research  Assistant 

Nearest  person  month  worked: 

i 

Contribution  to  Project: 

Ms.  Ananthram  supported  Ms.  Shewbridge  on  all  research  efforts. 

Name: 

James  Drane 

Project  Role: 

Mechanical  Engineer 

Nearest  person  month  worked: 

ii 

Contribution  to  Project: 

Mr.  James  Drane  assisted  in  the  coordination  of  the  team,  project 
management,  technical  assistance,  integration,  and  mechanical 
engineering  support  for  DANA. 

Name: 

Jonathan  Farris 

Project  Role: 

Software  Engineer 

Nearest  person  month  worked: 

8 

Contribution  to  Project: 

Mr.  Farris  led  the  programming  and  testing  teams  and  coordinated 
programming.  He  assured  both  adherences  to  project  timeline  and 
for  milestones,  as  well  as  quality  assurance. 

Name: 

Patrick  Gookin 

Project  Role: 

Software  Engineer 

Nearest  person  month  worked: 

i 

Contribution  to  Project: 

Assisted  Mr.  Farris  in  software  engineering  tasks. 

Name: 

Clifford  Knoll 

Project  Role: 

Software  Engineer 

Nearest  person  month  worked: 

10 

Contribution  to  Project: 

Mr.  Knoll  contributed  Android  programming  expertise  and  worked 
on  DANA  software  development. 

Name: 

Joli  Rightmyer 

Project  Role: 

Mechanical  Engineer 

Nearest  person  month  worked: 

i 

Contribution  to  Project: 

Ms.  Rightmyer  provided  integration  support  for  DANA  as  well  as 
testing  and  evaluation  of  technical  functionality. 

Final  Report  (W81XWH-12-C-0204  -  092712-001) 


17 


Name: 

Dr.  Emma  Roach 

Project  Role: 

Research  Scientist 

Nearest  person  month  worked: 

4 

Contribution  to  Project: 

Dr.  Roach  conducted  data  analysis  and  paper  writing. 

Name: 

Kyle  Rohrbach 

Project  Role: 

Mechanical  Engineer 

Nearest  person  month  worked: 

5 

Contribution  to  Project: 

Mr.  Rohrbach  provided  testing  and  evaluation  of  technical 
functionality. 

Name: 

Dr.  Clementina  Russo 

Project  Role: 

Research  Scientist 

Nearest  person  month  worked: 

5 

Contribution  to  Project: 

Dr.  Russo  conducted  data  analysis  and  paper  writing. 

Name: 

Charlotte  Safos 

Project  Role: 

Program  Director 

Nearest  person  month  worked: 

6 

Contribution  to  Project: 

Ms.  Safos  supported  Dr.  Lathan  in  the  coordination  of  staff  and 
overseeing  general  project  and  study  management. 

Name: 

Alicia  Salvino 

Project  Role: 

Research  Assistant 

Nearest  person  month  worked: 

2 

Contribution  to  Project: 

Ms.  Salvino  supported  Ms.  Shewbridge  on  research  efforts. 

Name: 

Rita  Shewbridge 

Project  Role: 
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Original  Articles 


Authors  alone  are  responsible  for  opinions  expressed  in  the  contribution  and  for  its  clearance  through 

their  federal  health  agency,  if  required. 


MILITARY  MEDICINE,  178,  4:365,  2013 


Defense  Automated  Neurobehavioral  Assessment  (DANA) — 
Psychometric  Properties  of  a  New  Field-Deployable 
Neurocognitive  Assessment  Tool 

Corinna  Lathan,  PhD*;  James  L.  Spira,  PhD,  MPHf;  Joseph  Bleiberg,  PhDf; 

Jack  Vice*;  C APT  Jack  W.  Tsao,  USN§ 


ABSTRACT  The  Defense  Automated  Neurobehavioral  Assessment  (DANA)  is  a  new  neurocognitive  assessment  tool 
that  includes  a  library  of  standardized  cognitive  and  psychological  assessments,  with  three  versions  that  range  from  a  brief 
5-minute  screen  to  a  45-minute  complete  assessment.  DANA  is  written  using  the  Android  open-source  operating  system 
and  is  suitable  for  multiple  mobile  platforms.  This  article  presents  testing  of  DANA  by  224  active  duty  U.S.  service 
members  in  five  operationally  relevant  environments  (desert,  jungle,  mountain,  arctic,  and  shipboard).  DANA  was  found 
to  be  a  reliable  instrument  and  compared  favorably  to  other  computer-based  neurocognitive  assessments.  Implications  for 
using  DANA  in  far-forward  military  settings  are  discussed. 


INTRODUCTION 

In  January  2009,  the  U.S.  Navy  Bureau  of  Medicine  and  Sur¬ 
gery  identified  a  need  to  enhance  existing  battlefield  concussion 
assessment  and  requested  the  development  of  a  durable,  porta¬ 
ble,  and  field-hardened  neurocognitive  assessment  tool  (NCAT) 
to  provide  a  practical  means  to  conduct  neurocognitive  and 
psychological  assessment  in  field  deployment  settings.  The  pur¬ 
pose  of  combining  neurocognitive  and  psychological  assess¬ 
ment  was  to  permit  more  comprehensive  evaluation  of  the 
broad  range  of  problems  that  may  be  encountered  during  com¬ 


*AnthroTronix,  8737  Colesville  Road,  Suite  L-203,  Silver  Spring,  MD 
20910. 

fDepartment  of  Veterans  Affairs,  National  Center  for  PTSD,  Pacific 
Islands  Health  Care  System,  3375  Koapaka  Street,  Honolulu,  HI  96819. 

^National  Intrepid  Center  of  Excellence,  Walter  Reed  National  Military 
Medical  Center,  8901  Wisconsin  Avenue,  Building  51,  Bethesda,  MD  20889. 

§Wounded,  Ill  and  Injured  Directorate,  US  Navy  Bureau  of  Medicine 
and  Surgery,  7700  Arlington  Boulevard,  Falls  Church,  VA  22032. 

The  preliminary  data  of  this  article,  “Defense  Automated  Neurobehavioral 
Assessment  (DANA):  A  Field-Deployable  Tool  for  Assessing  Concussion  and 
Deployment  Stress,”  was  presented  as  an  oral  presentation  by  the  authors 
J.L.S.,  C.L.,  J.B.,  and  J.W.T.  at  the  Military  Health  System  Research  Sympo¬ 
sium  (MHSRS)  Conference,  Fort  Lauderdale,  FL,  August  2012. 

The  opinions  or  assertions  contained  herein  are  the  private  views  of  the 
authors  and  are  not  to  be  construed  as  official  or  as  reflecting  the  views  of  the 
Department  of  Veterans  Affairs,  the  Department  of  the  Navy,  the  Depart¬ 
ment  of  Defense,  or  the  U.S.  Government. 

doi:  10.7205/MILMED-D-12-00438 


bat  deployment.  This  article  describes  the  resulting  NCAT, 
Defense  Automated  Neurobehavioral  Assessment  (DANA); 
DANA’s  psychometric  properties  based  on  assessment  of 
224  active  duty  U.S.  service  members  under  challenging  field 
conditions;  and  presents  comparisons  to  published  NCAT  data. 

DANA  consists  of  three  test  batteries  of  different  durations 
and  compositions  designed  for  increasingly  detailed  assess¬ 
ment  (Table  I).  The  three  batteries  include  (1)  DANA  Rapid, 
a  5 -minute  battery  of  three  basic  reaction-time  measures; 
(2)  DANA  Brief,  a  15 -minute  test  that  includes  DANA  Rapid 
plus  additional  neurocognitive  tests  as  well  as  psychological 
screening  tools  for  post-traumatic  stress  disorder  (PTSD), 
depression,  and  insomnia;  and  (3)  DANA  Standard,  a  45 -minute 
more  comprehensive  battery  of  neurocognitive  and  psycholog¬ 
ical  tests.  This  hierarchical  set  of  batteries  is  designed  to  facil¬ 
itate  health  care  providers’  access  to  standardized,  reliable,  and 
valid  objective  and  subjective  measures.  DANA’s  portability, 
multiple  test  batteries,  and  user-friendly  interface  enable  its 
use  by  a  wide  range  of  health  care  providers,  from  frontline 
medics/corpsmen  to  licensed  health  care  professionals. 

Establishing  reliability  and  feasibility  of  this  platform  in  a 
military  population  is  necessary  before  attempting  clinical 
validation  and  utilization.  The  eventual  goal  of  DANA  is  to 
assist  clinicians  to  (a)  make  rapid  and  accurate  assessment  of 
cognitive  and  psychological  dysfunction  secondary  to  brain 
injury  and/or  the  psychological  wounds  and  stressors  of  war, 
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Defense  Automated  Neurobehavioral  Assessment 


TABLE  I.  DANA  Test  Batteries* 


DANA  Rapid  (5  Minutes) 

DANA  Brief  (15  Minutes) 

DANA  Standard  (45  Minutes)** 

Simple  Reaction  Time  (SRT) 

SRT 

SRT 

Procedural  Reaction  Time  (PRO) 

Code  Substitution  Simultaneous  (CDS) 

CDS 

Go/No-Go  (GNG) 

PRO 

PRO 

Spatial  Discrimination  (SPD) 

SPD 

GNG 

GNG 

Code  Substitution  Delayed  (CDD) 

CDD 

SRT 

Matching  to  Sample  (MSP) 

Patient  Health  Questionnaire  (PHQ) 

Sternberg  Memory  Search  (STN) 

Primary  Care  PTSD  Screen  (PC-PTSD) 

SRT 

Insomnia  Screening  Index  (ISI) 

Combat  Exposure  Scale  (CES) 

PHQ 

Pittsburgh  Sleep  Quality  Index  (PSQI) 

PTSD  Checklist — Military  Version  (PCL-M) 
Deployment  Stress  Inventory  (DSI) 

*For  detailed  test  descriptions,  see  Table  AI.  **MSP  and  STN  were  still  under  development  at  the  time  of  this  testing  and  so  are  not  included  in  the  results. 


(b)  facilitate  referral  to  treatment  for  wounded  service 
members,  (c)  monitor  recovery,  and  (d)  aid  in  retum-to-duty 
determination.  Thus,  DANA  is  intended  to  enhance  military 
capability  and  better  ensure  a  healthy  fighting  force. 

METHODS 
DANA  Platform 

DANA  is  a  Java-based  mobile  application  that  runs  on  an 
Android  operating  system.  The  primary  advantages  of  Android 
are  that  it  is  open  source,  open  license  (Apache  2.0),  well 
supported  and  based  on  a  Linux  kernel,  which  is  nearly  ubiqui¬ 
tous.  Java  has  the  advantage  of  being  a  high-level,  class-based, 
object-oriented  language  designed  as  a  “write  once,  run  any¬ 
where”  solution  and  thus  is  portable  across  a  wide  range  of 
devices  and  desktops.  DANA,  therefore,  can  mn  on  any  Android 
mobile  device  and  can  be  used  with  a  stylus  or  touch  screen. 

Based  on  the  Navy  Bureau  of  Medicine  and  Surgery,  require¬ 
ments  for  MIL-SPEC  commercial-off-the-shelf  hardware, 
we  conducted  a  comprehensive  trade  study  and  selected  the 
Trimble  Nomad,  the  military-grade-hardened  handheld  com¬ 
puter  used  in  the  current  study.  A  Tektronix  100  MHz  analog 
to  digital  (ADC)  oscilloscope  was  used  to  test  the  input  variabil¬ 
ity  of  device  hardware  and  device  software  that  could  contribute 
to  the  overall  response  times.  A  push  action  switch  was 
connected  to  the  ADC,  which  was  then  used  as  the  input  stylus 
on  the  Nomad  to  measure  RT.  The  interval  between  two  inputs 
as  recorded  by  DANA  and  by  the  ADC  was  compared  over 
10  trials.  The  average  difference  was  6.8  milliseconds  with 
a  standard  deviation  (SD)  of  3.7  milliseconds.  By  comparison, 
the  input  variability  with  a  Microsoft  windows  personal  com¬ 
puter  can  range  from  4-25  milliseconds.1,2 

DANA  Test  Battery 

Selection  of  the  neurocognitive  and  psychological  tests 
included  in  DANA  was  established  by  a  tri-service,  Veterans 
Administration,  and  civilian  scientific  advisory  board  that 
included  military  and  civilian  neuropsychologists  and  psy¬ 


chologists,  neurologists,  and  corpsmen.  All  tests  included 
in  DANA’s  test  batteries  meet  the  requirements  of  the 
American  Psychological  Association’s  standard  for  tests  and 
measurements  and  all  tests  are  in  the  public  domain.  Eight 
cognitive  tests  and  seven  psychological  questionnaires  were 
selected  (Table  AI)  and  are  divided  into  three  test  batteries, 
as  shown  in  Table  I.  Tests  were  selected  based  upon  their 
potential  to  address  specific  deployment-related  concerns, 
such  as  concussion  and  combat  distress  or  exhaustion. 
Although  all  tests  utilized  have  an  extensive  literature  regard¬ 
ing  their  reliability  and  validity,  they  have  not  been  reported 
in  this  specific  configuration  nor  implemented  for  service 
members  in  this  manner.  The  advisory  board  also  contributed 
to  and  provided  feedback  on  the  user  interface  design  to  ensure 
ease  of  use  by  multiple  levels  of  caregivers  including  the 
corpsmen,  general  medical  officers,  and  neuropsychologists. 

Participants  and  Procedure 

To  evaluate  the  deployment  feasibility  of  DANA,  we  recruited 
224  active  duty  service  members  comprising  40  or  more  active 
duty  military  personnel  in  each  of  5  diverse  operational  envi¬ 
ronments.  No  subjects  were  excluded,  since  all  service  mem¬ 
bers  were  fit  for  duty,  not  undergoing  any  disability  evaluation, 
and  thus  assumed  to  be  healthy.  The  purpose  of  assessing 
service  members  across  diverse  environmental  conditions  was 
to  show  the  robustness  of  the  hardware  and  software  adminis¬ 
tration  under  different  operational  tempos  and  to  identify  any 
environmental  concerns  with  the  reliability  of  the  instmment. 

—  Arctic  (Thule  Air  Force  Base — Greenland  in  the  winter) 

—  Jungle  (U.S.  Marine  Corps  Jungle  Warfare  Training 
Center — Okinawa,  Japan,  in  the  summer) 

—  Altitude  (U.S.  Marine  Corps  Mountain  Warfare  Train¬ 
ing  Center — Bridgeport,  CA,  approximately  3,000  m) 

—  Desert  (U.S.  Marine  Corps  Desert  Warfare  Training 
Center — Twentynine  Palms,  CA,  in  the  summer) 

—  Shipboard  (USS  George  Washington  during  high  seas 
in  the  Western  Pacific) 
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Device  performance  (e.g.,  battery  life,  display  characteristics) 
was  evaluated  under  the  specific  environmental  conditions  (e.g., 
humidity,  temperature)  through  a  minimum  of  12-hour  expo¬ 
sure.  The  only  instrumentation  reliability  issue  was  a  screen 
refresh  rate  delay  in  the  Go/No-Go  (GNG)  test  in  the  arctic 
environment.  Because  this  screen  rendering  delay  would  affect 
test  results,  the  rendering  process  software  was  redesigned, 
which  successfully  mitigated  the  delay. 

The  research  protocol  was  approved  by  the  AnthroTronix 
Institutional  Review  Board,  the  VA  Institutional  Review  Board, 
and  received  a  Department  of  the  Navy  Human  Research  Pro¬ 
tections  Program  review.  A  letter  was  obtained  from  the  com¬ 
manding  officer  of  each  test  facility  and  all  subjects  signed  an 
informed  consent  document  to  participate  in  testing.  On  Day  1 , 
each  subject  was  tested  on  all  three  batteries,  the  DANA  Rapid, 
Brief,  and  Standard.  Subjects  returned  on  Day  2  to  repeat  the 
sequence  of  batteries  resulting  in  the  following  protocol: 

—  Day  1  (approximately  120  minutes) — Consent  Process, 
DANA  Rapid,  DANA  Brief,  DANA  Standard 

—  Day  2  (approximately  40  minutes) — DANA  Rapid, 
DANA  Brief,  DANA  Standard  (cognitive  tests  only) 


(The  above  times  include  5-minute  breaks  between 
each  battery.) 

A  research  team  of  clinical  psychologists  and  technical  staff 
administered  testing.  Participants  were  instructed  to  hold  the 
stylus  about  1-cm  above  the  screen,  and  to  respond  as  rapidly 
and  accurately  as  possible.  All  other  instmctions  were  embed¬ 
ded  within  the  tests.  To  minimize  learning  and  practice  effects, 
test  stimuli  are  generated  at  random  and  each  test  has  practice 
trials  before  the  actual  test  trials.  Parameters  of  the  final  ver¬ 
sion  of  DANA’s  individual  sub  tests  are  described  in  Table  All 
including  each  subtest’s  stimulus  presentation  duration, 
response  interval,  and  interstimulus  interval. 

Data  were  analyzed  in  SPSSv20  for  descriptive  statis¬ 
tics,  split-half  reliability,  test-retest  reliability,  and  cross-test 
correlations.  For  internal  consistency,  we  examined  split-half 
correlations  of  the  first  and  second  half  set  of  trials  for  the 
first  administration  of  each  test  on  each  day.  To  evaluate  test- 
retest  reliability  across  administrations,  we  calculated  intra¬ 
class  correlation  coefficients  (ICCs)3  that  have  been  used 
to  evaluate  reliability  for  other  health  status  instruments.4 
Because  of  multiple  analyses,  significance  levels  were  set  to 
between  p  <0.01  and  p  <0.001,  depending  upon  the  number 


TABLE  II.  Descriptive  Statistics  for  all  DANA  Variables  for  Each  Administration 


Task 

Administration 

n 

Median  RT  Correct  ±  SD 

Average  of  Median  Throughput  ±  SD 

Percentage  Correct  ±  SD 

1 

SRT 

1 

223 

309.7 

±65.3 

199.6 

±33.4 

99.7 

±3.3 

2 

223 

309.3 

±64.6 

199.8 

±34.6 

99.5 

±2.6 

3 

220 

300.6 

±55.5 

204.4 

±33.8 

99.4 

±3.4 

4 

213 

302.0 

±53.3 

202.2 

±33.4 

99.3 

±4.0 

5 

212 

308.3 

±65.1 

200.6 

±36.7 

99.3 

±4.7 

6 

172 

298.4 

±68.5 

207.1 

±31.7 

99.7 

±2.4 

7 

111 

307.8 

±77.0 

202.1 

±35.7 

99.4 

±2.3 

8 

164 

305.1 

±49.2 

200.7 

±30.6 

99.6 

±2.2 

9 

122 

317.9 

±69.0 

195.7 

±37.5 

99.7 

±1.5 

10 

121 

310.7 

±54.9 

197.8 

±30.7 

99.8 

±0.8 

2 

PRO 

1 

224 

604.5 

±101.6 

100.1 

±15.5 

98.3 

±4.3 

2 

220 

579.6 

±91.4 

103.8 

±15.1 

98.1 

±4.0 

3 

213 

571.8 

±84.8 

104.8 

±14.2 

98.0 

±4.0 

4 

174 

579.1 

±79.9 

103.6 

±13.4 

98.3 

±3.1 

5 

164 

579.7 

±95.4 

104.1 

±15.6 

98.2 

±3.5 

6 

122 

565.0 

±84.1 

105.7 

±15.0 

97.6 

±5.3 

3 

GNG 

1 

214 

535.4 

±96.8 

114.2 

±18.5 

99.1 

±2.5 

2 

214 

519.3 

±86.8 

117.6 

±18.5 

99.3 

±2.3 

3 

193 

520.0 

±91.4 

116.7 

±19.9 

98.2 

±4.5 

4 

163 

527.2 

±97.4 

116.1 

±19.2 

99.0 

±2.8 

5 

99 

506.4 

±98.3 

120.9 

±20.8 

98.8 

±3.6 

6 

75 

521.0 

±109.6 

117.7 

±21.7 

98.4 

±5.2 

4 

SPD 

1 

221 

1690.2 

±376.3 

34.4 

±7.1 

92.8 

±5.4 

2 

209 

1533.0 

±361.5 

37.3 

±9.4 

90.6 

±5.7 

3 

172 

1562.3 

±383.9 

37.8 

±9.6 

93.1 

±5.8 

4 

119 

1456.6 

±311.4 

38.2 

±7.4 

89.3 

±6.2 

5 

CDS 

1 

223 

1284.9 

±277.7 

47.5 

±9.5 

97.5 

±3.1 

2 

212 

1256.7 

±234.5 

47.8 

±8.9 

96.8 

±4.0 

3 

171 

1228.7 

±257.0 

49.8 

±10.8 

97.7 

±3.1 

4 

120 

1193.4 

±211.1 

50.5 

±9.4 

97.3 

±3.2 

6 

CDD 

1 

212 

1046.7 

±221.5 

55.0 

±11.9 

92.1 

±8.2 

2 

190 

1004.3 

±183.1 

56.2 

±11.7 

91.1 

±9.1 

3 

161 

996.7 

±184.2 

56.8 

±12.8 

90.9 

±9.5 

4 

103 

956.9 

±157.4 

59.6 

±11.5 

92.5 

±7.9 
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TABLE  III.  Comparison  to  Previously  Reported  Dataa 


DANA 

ANAM  (2006) 

ANAM  (2008) 

ANAM  (2012) 

SRT 

n 

223 

2,261 

5,237 

107,413 

Mean" 

309.7 

261.3 

267 

261 

SD 

65.3 

56.1 

74 

47 

Ratio 

21.08% 

21.47% 

27.72% 

18.01% 

PRO 

n 

224 

— 

— 

107,353 

Mean" 

604.5 

— 

— 

592 

SD 

101.6 

— 

— 

90 

Ratio 

16.81% 

— 

— 

15.20% 

CDS 

n 

223 

2,331 

5,237 

107,546 

Mean" 

1284.9 

1,191 

1,096 

1,162 

SD 

277.7 

248.7 

265 

272 

Ratio 

21.61% 

20.88% 

24.18% 

23.41% 

CDD 

n 

212 

1,891 

5,202 

107,523 

%  Accuracy 

92.1 

88.7 

86.30 

90 

SD 

8.2 

9.3 

15.80 

11.4 

Ratio 

8.90% 

10.48% 

18.31% 

12.67% 

"Medians  are  shown  for  DANA. 


of  analyses  conducted  to  correct  for  type-1  error.  Psycholog¬ 
ical  measures  were  scored  using  conventional  methods. 

RESULTS 

DANA  performed  well  in  all  five  field  environments  with  no 
significant  difference  across  data  sets;  therefore,  the  data  for 
all  five  operational  environments  were  combined.  The  num¬ 
ber  of  total  subjects  for  each  test  ranged  from  n  =  75  to  n  = 
224  depending  on  whether  or  not  the  service  member  was 
available  to  participate  in  all  administrations  across  2  days. 
All  but  one  test  (Code  Substitution  Delayed  [CDD])  had  over 
200  subjects  for  at  least  two  test  administrations. 

Scores  on  psychological  measures  revealed  an  overall  psy¬ 
chologically  healthy  sample.  Combat  Exposure  Scale  (CES) 
was  in  the  light  range  (3.7,  with  17  indicating  moderate  expo¬ 
sure).  PTSD  Checklist — Military  Version  (PCL-m)  (26), 
Patient  Health  Questionnaire  (PHQ)-8  (4.6),  Pittsburgh  Sleep 
Quality  Index  (PSQI)  (5.9),  and  Deployment  Stress  Inventory 
(DSI)  (9.7)  were  each  far  below  the  score  needed  to  reach 
clinical  criteria. 

Data  exclusions  included  the  elimination  of  trials  with 
anticipatory  responses  and  test  administrations  indicating 
random  responses  (criterion  for  exclusion  was  set  at  less  than 
65%  correct — since  tasks  are  binary,  50%  correct  represents 
random  responding).  No  slow  RT  responses  were  eliminated, 
but  to  mitigate  their  undue  influence  we  used  medians  rather 
than  means  to  describe  the  data.  Less  than  1  %  of  the  response 
trials  and  less  than  2%  of  the  test  administrations  were  elim¬ 
inated  based  on  these  criteria.  No  subjects  were  eliminated 
from  analysis  based  upon  a  criterion  of  having  more  than  one 
test  administration  eliminated  in  a  battery. 

Table  II  shows  the  descriptive  data  for  each  administration 
of  each  test,  and  Table  III  compares  DANA  median  RT  and  SD 
to  previously  published  reports  of  mean  data  from  the  Auto¬ 
mated  Neuropsychological  Assessment  Metrics  (ANAM, 
currently  used  by  the  U.S.  Department  of  Defense  for  baseline, 


predeployment  neurocognitive  testing)  in  military  person¬ 
nel.5-7  DANA  uses  median  correct  RT  as  the  relevant  metric, 
whereas  ANAM  reports  means;  however,  because  of  the  sam¬ 
ple  size  and  the  normal  populations  assessed,  the  mean  and 
median  are  assumed  to  be  similar  for  ANAM. 

Although  not  significantly  different,  absolute  RT  values 
may  be  different  because  of  differing  instrumentation  (stylus 
versus  mouse  button)  between  ANAM  and  DANA.  It  is  also 
possible  to  use  the  published  ANAM  data  to  calculate  Coef¬ 
ficients  of  Variation  (CVs)  (ratio  of  RT  to  SD  of  RT)  to  show 
stability8  and  these  are  compared  to  analogous  DANA  sub¬ 
tests  in  Table  III. 

For  reliability  within  administrations,  split-half  correla¬ 
tions  of  the  odd-even  trials  are  reported  for  the  first  adminis¬ 
tration  of  day  1  and  day  2  for  each  test.  Correlations  were 
acceptably  high  ( p  <  0.001)  and  generally  above  0.85 — Simple 
Reaction  Time  (SRT)  (0.91,  0.93),  Procedural  Reaction  Time 
(PRO)  (0.87,  0.86),  GNG  (0.85,  0.85),  Code  Substitu¬ 
tion  Simultaneous  (CDS)  (0.94,  0.93) — except  for  Spatial 
Discrimination  (SPD)  (0.76,  0.76)  and  CDD  (0.76,  0.82).  For 
test  reliability  across  administrations,  the  ICC  was  calculated. 
The  ICC  for  SRT  was  0.95  indicating  excellent  reliability 
across  the  10  administrations  over  2  days.  For  PRO,  GNG, 
SPD,  and  CDS,  similar  reliability  was  achieved  with  ICC 
values  of  0.91,  0.95,  0.89,  and  0.88,  respectively.  Only  the 
CDD  test  did  not  have  high  reliability  (0.54),  which  is 
expected  for  repeat  CDD  tests  within  a  short  time  period 
because  of  the  change  in  codes  with  each  administration, 
resulting  in  proactive  memory  interference. 

Correlations  were  also  conducted  between  all  psycholog¬ 
ical  and  cognitive  measures.  With  the  exception  of  CDD 
(difficulties  using  multiple  administrations  of  this  subtest 
because  of  memory  interference  were  noted  above),  all  cog¬ 
nitive  subtests  ( p  <  0.001)  and  psychological  tests  ( p  <  0.001) 
were  correlated  with  each  other;  however,  the  psychological 
and  cognitive  tests  did  not  correlate  with  each  other  in  this 
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TABLE  Al.  DANA  Test  Descriptions 


Test  Name  (Abbreviation) 

Task  Structure 

Task  Purpose 

Simple  Reaction  Time  (SRT)a-c 

The  subject  taps  on  the  location  of  the  yellow  asterisk 
symbol  as  quickly  as  possible  each  time  it  appears. 

This  task  measures  pure  RT. 

Procedural  Reaction  Time  (PRO)a  c 

The  screen  displays  one  of  four  numbers  for 

3  seconds.  The  subject  presses  on  a  left  button 
(“2”  or  “3”)  or  right  button  (“3”  or  “4”) 
depending  upon  the  number  pressed. 

A  choice  RT  measure  of  accuracy,  RT, 
and  impulsivity.  This  choice  RT  task 
targets  simple  executive  functioning 
with  easy  decision-making  capabilities. 

Go/No-Go  (GNG)a-c 

This  is  a  forced  choice  RT  task  relevant  to  warfighters. 

A  house  is  presented  on  the  screen  with  several 
windows.  Either  a  “friend”  (green)  or  “foe”  (white) 
appears  in  a  window.  The  respondent  must  push  a 
“fire”  button  only  when  a  “foe”  appears. 

A  choice  RT  measure  of  sustained 
attention  and  impulsivity.  The  test 
assesses  speed  and  accuracy  of  targets, 
omissions,  and  commissions. 

Spatial  Discrimination  (SPD)&,C 

Pairs  of  four-bar  histograms  are  displayed  on  the 
screen  simultaneously,  and  the  subject  is  requested 
to  determine  whether  they  are  identical.  One 
histogram  is  always  rotated  either  ±90  degrees 
with  respect  to  the  other  histogram. 

Assesses  visuospatial  analytic  ability. 

Code  Substitution  Simultaneous  (CDS)6,C 

Subjects  refer  to  a  code  set  of  9  symbol-digit  pairs 
that  are  shown  across  the  upper  portion  of  the 
screen.  A  sequence  of  single  symbol-digit  pairs 
is  shown  below  the  key,  and  the  subject  indicates 
whether  or  not  the  single  pair  matches  the  code 
by  pressing  Yes  or  No. 

Assesses  visual  scanning  and  attention, 
learning,  and  immediate  recall. 

Code  Substitution  Delayed  (CDD)&,C 

After  a  delay  of  several  intervening  tests,  the  same 
symbol-digit  pairs  are  presented  without  the  code. 

The  subject  indicates  whether  or  not  the  pairing 
was  included  in  the  code  that  was  presented  in  the 
earlier  code  substitution  learning  section. 

Assesses  learning  and  short-term  memory. 

Sternberg  Memory  Search  (STN)C 

The  subject  memorizes  a  set  of  five  letters,  after 
which  letters  appear  on  the  screen  one  at  a  time, 
and  the  subject  determines  if  the  letter  on  the 
screen  is  a  member  of  the  memory  set. 

Assesses  working  memory. 

Matching  to  Sample  (MSP)C 

A  single  4x4  checkerboard  pattern  is  presented  on 
the  screen  for  brief  study  period.  It  then  disappears 
for  5  seconds,  after  which  two  patterns  are  presented 
side-by-side.  The  subject  indicates  which  of  these 
two  patterns  matches  the  first. 

A  measure  of  short-term  memory, 
attention,  and  visuospatial 
discrimination. 

Insomnia  Screening  Index  (ISI)& 

A  5 -item  scale  evaluating  perceived  insomnia  severity 
and  sleep  habits.  Each  item  is  rated  on  a  5 -point 
scale  (0-4). 

The  total  score  ranges  from  0  to  28  and 
higher  scores  indicate  more  severe 
insomnia.  A  cutoff  score  of  10  has 
been  shown  to  indicate  insomnia.11 

Primary  Care  PTSD  Screen 

Four  screening  questions  designed  for  use  in  clinical 

Questions  assess  hyper-arousal, 

(PC-PTSD)^ 

settings  to  screen  for  PTSD,  with  3  out  of  4  endorsed 
items  suggestive  of  likely  PTSD. 

re-experiencing,  and  avoidance  for 

PTSD  screening.  This  test  is  more 
sensitive  than  specific,  but  correlates 
highly  with  the  PCL. 12,13 

Patient  Health  Questionnaire  (PHQ)^,C 

A  9-item  depression  scale  assessing  symptom  severity 
and  diagnostic  criteria  for  major  depressive  disorder. 
For  research  purposes,  item  no.  9  (concerning 
suicide)  was  not  included,  yet  research  indicates 
that  the  scoring,  reliability,  and  clinical  validity  are 
almost  identical. 

A  score  of  0-9  is  likely  to  have  no 
depression,  10-14  mild  depression, 

15-19  moderate  depression,  and  20+ 
severe  depression.14 

Pittsburgh  Sleep  Quality  Index 

19  self-rated  items  and  5  partner-rated  items,  which 

This  scale  is  the  most  widely  utilized 

(PSQI)C 

measure  sleep  quality  during  the  previous  month. 

This  scale  differentiates  “good”  from  “poor” 
sleepers  based  on  seven  areas:  subjective  sleep 
quality,  sleep  latency,  sleep  duration,  habitual 
sleep  efficiency,  sleep  disturbances,  use  of 
sleeping  medication,  and  daytime  dysfunction 
over  the  last  month. 

sensitive  and  specific  self-report 
measure  for  insomnia.  A  score  above 

6  indicates  a  “poor”  sleeper,  and  a 
score  above  12  is  associated  with 
“insomnia.”15 

( continued ) 
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TABLE  Al.  Continued 


Test  Name  (Abbreviation) 

Task  Structure 

Task  Purpose 

Combat  Exposure  Scale  (CES)C 

A  7-item  self-report  measure  that  assesses  wartime  stressors 
experienced  by  service  members.  The  total  CES  score 
(ranging  from  0  to  41)  is  calculated  by  using  a  sum  of 
weighted  scores,  which  can  be  classified  into  1  of  5 
categories  of  combat  exposure  ranging  from  “light” 
to  “heavy.” 

This  scale  rates  cumulative  combat 
exposure  and  is  highly  predictive  of 

PTSD,  pain  and  injury,  TBI,  depression, 
and  other  behavioral  sequellae.16 

PTSD  Checklist — Military 

A  17-item  scale  assessing  symptoms  in  response  to  stressful 

Higher  scores  indicate  increased 

Version  (PCL-M)C 

military  experiences.  This  scale  assesses  PTSD,  with 
subscales  including  re-experiencing,  avoidance/numbing, 
and  hyperarousal. 

PTSD  symptomatology.  In  a  military 
population,  scores  >49  are  likely  to 
have  PTSD.  For  greatest  specifically, 
scores  >44  with  3  re-experiencing, 

1  avoidance/numbing,  and 

2  Hyperarousal  endorsed  as  at  least 
“most  of  the  time”  are  more  specific 
for  PTSD  and  correlate  very  highly 
(0.92)  with  the  Clinician  Administered 
PTSD  Scale  (CAPS).17 

Deployment  Stress  Inventory 

Based  upon  the  neurobehavioral  symptom  inventory  with 

This  experimental  measure  is  intended 

(DSI)C 

additional  items  added  to  assess  anger,  pain,  and  sleepiness. 
Test  is  a  28-item  experimental  scale  that  factors  into  three 
domains  (cognitive-emotional,  somatic,  and  anger)  and  five 
subscales  (cognitive,  emotional,  pain,  sleep,  and  anger). 

to  be  used  as  a  broad  psychological 
screening  tool  sensitive  to 
combat-related  distress,  especially 
reporting  of  persistent  postconcussive 
symptoms.18 

"DANA  Rapid  Battery.  /jDANA  Brief  Battery.  CDANA  Standard  Battery. 


sample  of  nonimpaired  service  members.  The  CES  correlated 
mildly  with  the  PCL-m  and  DSI  ( p  <  0.01). 

DISCUSSION 

The  data  presented  here  represent  a  first  empirical  examina¬ 
tion  of  the  DANA  tool,  a  portable  NCAT  that  includes  both 
cognitive  and  psychological  tests.  Feasibility,  reliability,  and 
internal  validity  were  assessed  through  the  administration 
of  DANA  to  224  active  duty  service  members  (officers  and 
enlisted)  from  the  U.S.  Air  Force,  Navy,  and  Marine  Corps 
across  five  extreme  environments. 

As  can  be  seen  in  Table  II,  scores  were  stable  across 
administrations.  Split-half  reliability  correlations  for  DANA 
subtests  are  within  acceptable  ranges,  and  are  comparable 
to  those  reported  for  other  similar  NCAT  subtests.9  ICC 


measures  across  administrations  were  excellent  and  generally 
exceeded  those  reported  in  similar  NCAT  subtests  although 
they  were  comparable  to  previously  reported  aggregate  NCAT 
scores.10  We  are  currently  developing  more  sophisticated 
scoring  and  statistical  approaches  to  score  each  subtest  as 
well  as  to  aggregate  subtests  into  composite  index  scores. 

Descriptive  analysis  of  the  psychological  subtests  shows 
that  the  sample  scored,  in  general,  well  below  scores  indica¬ 
tive  of  clinical  problems.  From  the  range  of  CES  scores,  it 
appears  that  most  of  the  sample  had  not  been  exposed  to 
moderate  or  greater  combat.  With  regard  to  mean  psycholog¬ 
ical  scores  and  the  relevant  scoring  for  cognitive  variables,  all 
psychological  variables  correlate  highly  with  each  other, 
with  the  exception  of  the  Insomnia  Severity  Index  (ISI)  and 
Primary  Care  (PC)-PTSD,  which  had  moderate  correlations, 
likely  due  to  having  a  low  number  of  items  (4  and  7  items, 


TABLE  AIL  Test  Parameters 


No.  of  Response  Trials 

Stimulus  Presentation  Time 
(milliseconds) 

Maximum  Response  Time 
(milliseconds) 

Intertrial  Interval 
(milliseconds) 

SRT 

40 

900 

900 

600  to  3,000 

PRO 

32 

2,000 

2,000 

500  to  1,000 

GNG 

30 

1,500 

1,500 

1,000  to  1,750 

CDS 

72 

3,000 

3,000 

900 

CDD 

72 

6,000 

6,000 

900 

SPD 

20 

5,000 

5,000 

500  to  1,000 

MSP" 

30 

3,000 

10,000 

750  to  1,350 

STN 

30 

5,000 

5,000 

900 

"MSP  also  had  a  delay  between  the  stimulus  and  the  response  grids  of  5,000  milliseconds. 
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respectively).  Similarly,  most  cognitive  tests  correlated  mod¬ 
erately  with  each  other,  with  the  exception  of  CDD,  which 
did  not  correlate  with  the  other  cognitive  measures,  likely 
tapping  into  a  different  construct  than  the  other  measures. 
In  addition,  in  this  nonclinical  population,  no  cognitive  tests 
correlated  with  any  psychological  tests.  This  is  to  be  expected 
given  that  the  great  majority  of  scores  on  the  psychological 
tests  were  well  within  the  “normal”  range,  and  the  cognitive 
scores  were  closely  grouped.  Finally,  CES  did  not  correlate 
with  any  measure  in  this  nonclinical  population  of  mostly 
noncombat  deployed  service  members. 

As  can  be  seen  in  Table  IE,  DANA  compares  favorably  to 
existing  NCATs  in  terms  of  median/mean  RT  (or  %  accuracy 
for  CDD)  and  SD.  The  CVs  are  also  consistent  with  CVs 
reported  from  ANAM  data  collected  in  2006,  2008,  and  2012 
cohorts.  This  suggests  that  differences  in  absolute  values  for 
RT  are  most  likely  due  to  the  testing  instrument  (mouse  click 
versus  stylus)  rather  than  anything  implicit  in  the  test  itself. 
Taken  together,  DANA  appears  to  have  adequate  reliability 
and  test  validity  in  a  sample  of  nonclinical  service  members 
across  services  and  environments.  DANA  is  currently  being 
assessed  in  postdeployment  and  clinical  samples. 

The  results  reported  here  suggest  that  the  DANA  has  prom¬ 
ise  as  a  next  generation  NCAT.  Benefits  of  the  DANA  include 
that  it  (a)  includes  relevant  psychological  tests  as  well  as  stan¬ 
dard  cognitive  tests;  (b)  is  built  on  a  portable  OS  with  more 
precise  timing  than  previous  NCATs;  and  (c)  is  a  portable 
handheld  device,  which  is  more  versatile  than  a  laptop  com¬ 
puter.  Future  studies  of  DANA  are  planned  to  assess  DANA’s 
ability  to  assist  frontline  providers  to  more  rapidly  and  accu¬ 
rately  evaluate  service  members  to  determine  the  need  for 
higher  level  evaluation,  treatment,  or  readiness  to  return  to  duty. 

APPENDIX 

Table  AI  describes  the  eight  cognitive  tests  and  seven  psy¬ 
chological  questionnaires  that  were  selected  for  the  DANA 
test  batteries.  Each  individual  subtest  parameters  are  shown 
in  Table  All  including  stimulus  presentation  duration, 
response  interval,  and  interstimulus  interval. 
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Abstract 

An  understanding  of  human  responses  to  hypoxia  is  important  for  the  health  of  millions  of  people  worldwide  who  visit,  live, 
or  work  in  the  hypoxic  environment  encountered  at  high  altitudes.  In  spite  of  dozens  of  studies  over  the  last  100  years,  the 
basic  mechanisms  controlling  acclimatization  to  hypoxia  remain  largely  unknown.  The  AltitudeOmics  project  aimed  to 
bridge  this  gap.  Our  goals  were  1)  to  describe  a  phenotype  for  successful  acclimatization  and  assess  its  retention  and  2)  use 
these  findings  as  a  foundation  for  companion  mechanistic  studies.  Our  approach  was  to  characterize  acclimatization  by 
measuring  changes  in  arterial  oxygenation  and  hemoglobin  concentration  [Hb],  acute  mountain  sickness  (AMS),  cognitive 
function,  and  exercise  performance  in  21  subjects  as  they  acclimatized  to  5260  m  over  16  days.  We  then  focused  on  the 
retention  of  acclimatization  by  having  subjects  reascend  to  5260  m  after  either  7  (n  =  14)  or  21  (n  =  7)  days  at  1525  m.  At  16 
days  at  5260  m  we  observed:  1)  increases  in  arterial  oxygenation  and  [Hb]  (compared  to  acute  hypoxia:  Pa02  rose 
9±4  mmHg  to  45±4  while  PaC02  dropped  a  further  6±3  mmHg  to  21  ±3,  and  [Hb]  rose  1.8±0.7  g/dL  to  16±2  g/dL;  2)  no 
AMS;  3)  improved  cognitive  function;  and  4)  improved  exercise  performance  by  8±8%  (all  changes  p<0.01).  Upon  reascent, 
we  observed  retention  of  arterial  oxygenation  but  not  [Hb],  protection  from  AMS,  retention  of  exercise  performance,  less 
retention  of  cognitive  function;  and  noted  that  some  of  these  effects  lasted  for  21  days.  Taken  together,  these  findings 
reveal  new  information  about  retention  of  acclimatization,  and  can  be  used  as  a  physiological  foundation  to  explore  the 
molecular  mechanisms  of  acclimatization  and  its  retention. 
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Introduction 

Millions  of  people  live  and  work  in,  or  travel  to,  high  altitudes, 
and  many  of  them  are  able  to  adjust  successfully  to  the  hypoxic 
environment  of  very  high  altitudes  (~5000  m),  where  ambient 
oxygen  pressure  is  about  half  the  sea  level  value.  Discovery  of  the 
mechanisms  responsible  for  human  acclimatization  to  hypoxia 


could  lead  to  new  ways  to  improve  acclimatization  and  its 
retention. 

The  physiology  of  how  humans  respond  acutely  and  adapt  to 
hypoxia  has  been  explored  extensively  over  the  last  century,  yet 
many  questions  remain  about  the  attributes  that  best  characterize 
acclimatization  [1].  Most  would  agree  that  improving  arterial 
oxygenation  and  exercise  performance  are  central  tenets  of 
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acclimatization,  and  although  no  studies  have  focused  on  the 
protection  from  high-altitude  illness  that  occurs  with  acclimatiza¬ 
tion,  most  would  also  agree  such  protection  is  an  important  aspect 
of  acclimatization.  On  the  other  hand,  how  cognitive  function 
responds  during  acclimatization  is  largely  unknown,  except  from 
anecdotal  reports.  Intriguing  also  are  suggestions  that  acclimati¬ 
zation  causes  functional  modifications  that  persist  upon  return  to 
high  altitude  after  weeks,  or  perhaps  even  months,  at  sea  level,  and 
at  a  time  when  all  known  physiological  measures  of  acclimatiza¬ 
tion  have  returned  to  normal  low  altitude  values  [2-4]. 

AltitudeOmics  is  a  multifaceted  research  program  on  acclima¬ 
tization  to  high  altitude  and  the  retention  of  acclimatization  after 
return  to  low  altitude.  The  goals  for  AltitudeOmics  were  1)  to 
describe  a  phenotype  for  successful  acclimatization  and  assess  its 
retention — that  is — whether  adaptive  responses  persist  after 
descent  to  low  altitude  for  one  to  three  weeks,  and  2)  to  use  these 
findings  as  a  foundation  for  companion  mechanistic  studies  of  the 
human  transcriptome,  epigenome,  metabolome,  and  proteome 
(OMIGS).  Our  approach  was  to  study  lowland  volunteers  in  the 
field  who  were  taken  rapidly  to  5260  m,  where  they  acclimatized 
for  16  days.  They  then  descended  to  1525  m  for  either  seven 
(n  =  14)  or  21  (n  =  7)  days,  after  which  they  returned  quickly  to 
5260  m  and  were  retested.  This  report  describes  the  physiology  of 
acclimatization  and  its  retention  for  four  key  features  of 
acclimatization:  1)  arterial  oxygenation  and  [Hb];  2)  acute 
mountain  sickness  (AMS);  3)  cognitive  function;  and  4)  exercise 
performance.  Of  particular  interest  was  the  acclimatization 
retention  displayed  upon  returning  to  5260  m  after  even  three 
weeks  at  low  altitude.  Subsequent  reports  will  explore  changes  in 
OMICS  responses  and  will  attempt  to  link  those  responses  to  the 
physiological  phenotype  of  acclimatization  and  its  retention 
reported  here. 

Methods 

Ethical  Approval  and  Subject  Recruitment 

The  study  was  performed  according  to  the  Declaration  of 
Helsinki.  It  was  approved  by  the  Institutional  Review  Boards  of 
the  Universities  of  Colorado  and  Oregon  and  by  the  Human 
Research  Protection  Office  of  the  U.S.  Department  of  Defense. 
The  subjects  were  informed  about  the  possible  risks  and 
discomforts  of  participation  in  the  study  before  giving  their  written 
and  verbal  consent  to  participate.  Physical  examinations  and  the 
U.S.  Army  Physical  Fitness  Test  (APFT)  (push-ups,  sit-ups,  and  a 
3.2-km  run)  [5]  were  performed  to  characterize  health  and  fitness 
status.  Exclusion  criteria  included:  being  born  at  >1500  m;  having 
traveled  to  altitudes  >1000  m  in  the  past  three  months  (including 
air  travel);  using  prescription  medications;  smoking;  being 
pregnant  or  lactating;  having  a  history  of  serious  head  injury  (loss 
of  consciousness);  self  or  familial  history  of  migraine;  known 
hematologic  or  cardiovascular  abnormality  (e.g.,  sickle  cell  trait, 
cardiac  arrhythmia);  pulmonary  function  or  diffusion  capacity  for 
carbon  monoxide  <90%  of  predicted;  or  failure  to  meet  the 
minimal  age/gender  standards  for  the  APFT  [5].  Seventy-nine 
subjects  completed  the  screening.  Twenty-four  healthy,  physically 
active  subjects  were  enrolled.  Two  subjects  dropped  out  for  non¬ 
altitude  related  medical  reasons,  and  one  was  never  healthy  at 
high  altitude  due  to  non-altitude  related  persistent  gastrointestinal 
illness.  Thus,  21  subjects  (12  males  and  nine  females,  average  age 
20.8  yrs,  range  19-23  yrs)  constitute  the  AltitudeOmics  group  of 
subjects  included  in  this  and  subsequent  reports  (Table  1). 

Timeline.  Each  subject  was  studied  near  sea  level  (SL) 
(130  m,  average  PB  =  749  mmHg,  Figure  1),  and  over  three  study 
periods  at  Mt  Chacaltaya,  Bolivia;  5260  m;  average 


Table  1.  General  Subject  Characteristics. 


ID 

Gender 

Age  (years) 

HT  (cm) 

WT  (kg) 

BMI  (kg/m2) 

001 

M 

22 

184.2 

80.8 

23.8 

002 

M 

22 

181.6 

65.4 

19.8 

003 

F 

21 

166.4 

54.3 

19.6 

004 

M 

21 

181.6 

70.7 

21.4 

005 

F 

21 

160.0 

53.2 

20.7 

006 

M 

19 

170.2 

68.1 

23.5 

007 

M 

21 

184.2 

73.3 

21.6 

010 

F 

19 

163.8 

67.6 

25.1 

011 

F 

21 

169.5 

68.0 

23.6 

012 

M 

20 

181.6 

82.4 

24.9 

013 

M 

23 

182.9 

77.0 

23.0 

014 

M 

21 

186.7 

85.4 

24.4 

015 

F 

22 

168.3 

56.7 

20.0 

017 

F 

23 

174.0 

69.9 

23.0 

018 

M 

21 

180.3 

79.9 

24.5 

019 

F 

19 

176.5 

68.0 

21.8 

020 

F 

19 

165.7 

62.2 

22.6 

021 

M 

20 

182.9 

68.9 

20.6 

022 

M 

23 

180.3 

73.8 

22.6 

023 

M 

20 

179.1 

77.8 

24.2 

025 

F 

19 

172.1 

60.9 

20.5 

Mean 

1 2M/9F 

20.8 

175.8 

69.7 

22.4 

SD 

1.4 

7.9 

9.0 

1.8 

Height  (HT);  Weight  (WT);  Body  Mass  Index  (BMI). 
doi:1 0.1 371/journal.pone.00921 91  ,t001 


PB  =  406  mmHg);  on  the  first/ second  and  sixteenth/ seventeenth 
days  at  5260  m  (ALT1,  ALT  16),  and  again  upon  reascent  to 
5260  m,  after  either  seven  (n  =  14)  or  21  (n  =  7)  days  at  low  altitude 
(POST7  or  POST21).  Baseline  studies  at  SL,  including  laboratory 
(physiologic  and  OMIGS)  and  field  (3.2-km  uphill  run)  tests,  were 
conducted  over  a  two-week  period  in  Eugene,  OR,  USA. 
Approximately  one  month  after  the  SL  studies,  subjects  traveled 
to  Bolivia  in  pairs  on  successive  days.  Upon  arrival  at  El  Alto 
(4050  m)  after  an  overnight  flight,  subjects  immediately  descended 
to  Goroico,  Bolivia  (1525  m;  PB  =  639  mmHg).  Subjects  rested  for 
48  hrs  in  Coroico  to  limit  the  effects  of  jet  lag  and  were  then 
driven  over  three  hrs  to  5260  m.  To  provide  an  acute  change  in 
inspired  P02  from  1525  m  to  5260  m,  subjects  breathed 
supplemental  oxygen  (2  L/min,  nasal  cannula  or  mask)  during 
the  drive.  On  arrival  at  5260  m,  the  first  subject  immediately 
began  the  experimental  protocol  described  below.  The  second 
subject  rested  while  continuing  to  breathe  supplemental  oxygen  for 
~  two  hrs  until  the  first  subject  had  completed  the  arterial/venous 
catheterization  and  cognitive  testing  portion  of  the  protocol.  Then 
the  second  subject  began  the  protocol  as  described  for  the  first 
subject.  Two  subjects  were  studied  per  day  for  ALT1,  ALT  16, 
POST7,  and  POST21.  After  completing  laboratory  testing  and 
AMS  scoring  on  ALT1,  subjects  slept  overnight  on  supplemental 
oxygen  to  minimize  the  risk  of  developing  severe  high-altitude 
illness.  The  next  morning,  subjects  completed  a  3.2-km  uphill  run 
(305  m  elevation  gain)  before  descending  by  car  to  La  Paz,  Bolivia 
(3800  m;  average  PB  =  487  mmHg)  to  continue  acclimatizing  at  a 
lower  altitude  over  three  nights  (ALT2-ALT4).  On  ALT4  subjects 
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visited  5260  m  for  four  to  six  hrs.  On  ALT5,  they  returned  to 
5260  m,  where  they  remained  for  an  additional  13  days.  On 
ALT16/17  subjects  were  tested,  as  on  ALT  1/2  prior  to 
descending  by  car  to  1525  m.  To  test  physiological  retention  of 
acclimatization  after  living  for  seven  (n  =  14)  or  21  (n  =  7)  days  at 
low  altitude  (1525  m),  subjects  returned  to  5260  m  by  car,  as  they 
did  on  ALT1  but  this  time  without  supplemental  oxygen,  and 
completed  the  POST7/21  testing  (detailed  below).  After  comple¬ 
tion  of  a  3.2-km  uphill  run  on  POST7/21,  the  subjects  returned 
home.  Assignment  to  POST7  or  POST2 1  was  determined  by  each 
subject  based  on  their  desire  to  stay  in  the  field  an  extra  seven  or 
2 1  days.  While  in  Bolivia,  subjects  were  housed  and  fed  as  a  group. 
Meals  and  snacks  were  kept  similar  to  the  subjects’  typical  ad 
libitum  diet.  Subjects  were  instructed  to  ingest  at  least  three  liters  of 
water  each  day  and  to  remain  physically  active. 

Experimental  Protocol.  Testing  progressed  in  the  following 
general  order:  1)  radial  artery  and  antecubital  vein  catheterization; 
2)  30-min  supine  rest,  followed  by  cognitive  function  testing;  3) 
measurement  of  resting  (seated)  arterial  blood  gases  and  hemo¬ 
globin  concentration,  and  blood  draw  for  OMICS  samples;  4) 
cycle  ergometry  exercise  testing;  5)  AMS  symptom  scoring;  and, 
on  a  separate  day,  6)  a  3.2-km  uphill  run.  In  addition  to  the  studies 
presented  here,  within  the  framework  of  AltitudeOmics  and 
reported  separately,  we  also  assessed  cerebral  blood  flow  [6]  and 
cerebral  autoregulation  [7];  chemical  control  of  breathing  [8]; 
total  hemoglobin  mass  and  blood  volume  compartments;  periph¬ 
erally  [9]  and  centrally  [10]  derived  measures  of  exercise-induced 
fatigue;  blood  flow  through  intracardiac  shunt  (patent  foramen 
ovale)  and  intrapulmonary  arteriovenous  anastomoses;  and 
OMICS  responses  (transcrip tomics,  epigenomics,  metabolomics, 
and  proteomics). 

Procedures 

Anthropometry.  Height  (cm)  was  measured  at  SL  only.  Body 
mass  (kg)  was  recorded  at  SL,  ALT1,  ALT  16,  and  POST7/21 


using  the  same  scale  (Seca  770,  Hanover,  MD,  USA),  with  the 
subject  wearing  light  underwear  and  no  shoes. 

Arterial  Blood  Gases  and  Hemoglobin 

Under  local  anesthesia  (2%  lidocaine)  a  20—22  G  radial  artery 
catheter  (Models  RA-04122/RA-04020  Arrow  International, 
Reading,  PA,  USA)  was  placed  for  the  duration  of  experiments 
conducted  at  SL,  ALT1/16,  and  POST7/21.  Arterial  blood 
samples  were  drawn  anaerobically  and  immediately  analyzed  in 
duplicate  for  Pa02,  PaC02,  pH  (Siemens  RAPIDLab  248, 
Erlangen,  Germany),  [Hb]  and  Sa02  (Radiometer  OSM3, 
Copenhagen,  Denmark).  Core  temperature  was  measured  using 
an  ingestible  temperature-sensing  pill  (CorTemp  HQInc,  Palmet¬ 
to,  FL,  USA)  [11,12].  Blood  gases  were  corrected  for  core 
temperature  [11,12].  Ca02  (mL/dL)  was  calculated  as: 
Ca02  =  1.39  *  [Hb]  *  (Sa02/100)+(Pa02  *  0.003).  The  Hill 
equation  was  used  to  calculate  P50  [13].  Resting  arterial  blood 
samples  were  taken  following  1 0  min  of  seated  rest  at  SL,  ALT  1 , 
ALT  16,  and  POST7/21. 

Acute  Mountain  Sickness 

The  severity  of  AMS  symptoms  was  assessed  using  the  Lake 
Louise  Questionnaire  (LLQ,  which  includes  a  self-reported 
assessment  of  AMS  symptoms  (headache,  fatigue,  gastrointestinal 
discomfort,  and  dizziness)  and  the  shortened  Environmental 
Symptom  Questionnaire  (AMS-C).  Total  LLQ  scores  that 
included  headache  and  were  ^3  or  >6  (out  of  a  possible  total 
of  12)  were  diagnostic  of  moderate  or  severe  AMS,  respectively. 
Quality  of  sleep  was  not  included  in  the  total  LLQ  score  because 
nights  prior  to  ALT1  and  POST7/21  were  spent  at  low  altitude. 
Recently,  in  our  laboratory,  we  have  published  LLQ  without  using 
the  sleep  question,  with  no  change  in  sensitivity  in  identifying 
AMS  [14,15].  AMS-C  is  a  self-reported  11 -question  inventory 
from  which  a  score  >  0.7  is  considered  indicative  of  AMS  [16]. 
AMS  symptoms  were  assessed  at  SL,  ALT1  (in  the  evening, 


Sea  1  qwqI 

Time  (days) 

Figure  1.  Timeline  for  AltitudeOmics  Studies.  Each  subject  completed  this  study  timeline,  with  n  =  14  staying  at  low  altitude  for  POST7  and 
n  =  7  staying  at  low  altitude  for  POST21.  Subjects  flew  from  the  USA  to  Bolivia  aboard  commercial  aircraft  with  no  recording  of  barometric  pressure 
during  the  flight;  the  profile  for  travel  in  the  figure  is  therefore  approximate. 
doi:1 0.1 371/journal.pone.00921 91  .g001 


PLOS  ONE  |  www.plosone.org 


3 


March  2014  |  Volume  9  |  Issue  3  |  e92191 


Human  Acclimatization  to  Hypobaric  Hypoxia 


10-12  hrs  after  arrival),  ALT  16,  and  POST7/21  (time-matched 
to  ALT1). 

Cognitive  Function 

The  Defense  Automated  Neurobehavioral  Assessment  (DANA) 
was  used  to  assess  neurocognitive  function.  DANA  is  a  neurocog- 
nitive  assessment  tool  that  includes  a  library  of  open-source, 
standardized,  cognitive  and  psychological  assessments  [17].  Using 
a  handheld  computer,  the  following  nine  cognitive  function  tests 
were  administered:  1)  Simple  Reaction  Time-1  (measured  at  the 
beginning  of  neurocognitive  testing  to  gain  an  understanding  of 
pure  visual-motor  response);  2)  Simple  Reaction  Time-2,  repeated 
at  the  end  of  neurocognitive  testing  to  assess  diminished  reserve  of 
cognitive  effort  on  reaction  time;  3)  Procedural  Reaction  Time,  a 
measure  of  choice  reaction  time  and  accuracy;  4)  Go-No-Go,  a 
measure  of  speed,  accuracy  and  impulsivity;  5)  Code  Substitu¬ 
tion — Simultaneous,  a  measure  of  visual  scanning  and  attention, 
learning,  and  immediate  recall  of  digit-symbol  pairings;  6)  Code 
Substitution — Delayed  Recall,  a  measure  of  short-term  memory 
for  digit-symbol  pairings;  7)  Spatial  Discrimination,  a  measure  of 
visuospatial  analytic  ability;  8)  Match  to  Sample,  an  assessment  of 
attention  and  memory  for  visuospatial  discrimination;  and  9) 
Sternberg’s  Memory  Search,  a  measure  of  working  memory  for 
letters.  Neurocognitive  tests  were  administered  before  and  after  the 
expedition  at  SL  and  once  each  at  ALT1,  ALT  16  and  POST7/21. 
Repeat  cognitive  function  tests  at  SL  were  similar  (p>0.5)  and  thus 
were  combined  to  give  one  SL  score  for  comparison  to  changes  in 
cognitive  function  at  5260  m.  Mean  throughput,  a  measure  of 
mental  efficiency,  is  calculated  as  the  mean  number  of  correct 
responses  for  each  test  made  within  one  min  [18]  and  is  the 
outcome  variable  reported  for  all  cognitive  function  variables. 

Exercise 

Laboratory  exercise  testing.  Incremental  exercise  tests  to  maximal 
exertion  on  an  electrically-braked  cycle  ergometer  (V elotron  Elite, 
Racermate,  Seattle,  WA,  USA)  were  used  to  assess  peak  aerobic 
power.  Subjects  completed  three-min  stages  at  70,  100,  130  and 
160  Watts,  followed  by  15  Watts/min  increments  until  they  could 
no  longer  maintain  pedaling  at  >  50  rpm.  Peak  aerobic  power 
(Watts)  was  calculated  as:  work  rate  of  last  stage  completed  + 
[(work  rate  increment)  *  (time  into  final  stage/ duration  of  stage  in 
seconds)]  [19].  Exercise  tests  were  performed  at  SL,  ALT1,  and 
ALT  16,  but  not  at  POST7/21  due  to  logistical  issues. 

Field  exercise  testing.  Subjects  completed  a  timed  3. 2 -km  uphill  run 
as  fast  as  possible,  on  unpaved  roads,  with  an  identical  elevation 
gain  of  305  m.  Tests  were  performed  at  SL  at  least  48  hrs  before 
the  laboratory  tests  and  in  the  morning  after  an  overnight  stay  on 
ALT1,  ALT  16  and  POST7/21.  Performance  was  expressed  as 
mean  running  speed  in  m/ s. 

Data  Analysis 

As  expected,  preliminary  analyses  revealed  higher  Ca02  for 
males  as  a  result  of  higher  [Hb],  across  the  study  (p<0.01  vs. 
females);  however,  since  the  sex  vs.  time  interaction  was  not 
significant  (p>0.05)  male  and  female  data  were  pooled  for  all 
subsequent  analyses.  Lor  physiological  variables,  paired  t-tests, 
with  Bonferroni  correction  for  multiple  testing,  were  completed  for 
comparisons  among  time  points.  LLQ  AMS-C  scores  and 
cognitive  function  tests  were  evaluated  by  the  Wilcoxon  signed 
rank  test.  The  Spearman  rank  order  and  Pearson  product  moment 
correlations  were  run  to  evaluate  associations  between  changes  in 
arterial  blood  gases  and  [Hb]  and  changes  in  AMS  symptoms, 
cognitive  function,  and  physical  performance  across  time.  Due  to 
transportation  delays  and  the  technical  challenges  inherent  to  field 
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studies,  not  all  procedures  were  completed  on  all  subjects  at  Mt. 
Ghacaltaya  (see  Tables  SI,  S2,  S3,  S4,  S5  for  respective  sample 
sizes).  Overall,  most  subjects  completed  most  tests,  with  88%  of 
arterial  blood  gas  and  hematology  measurements,  100%  comple¬ 
tion  of  AMS  and  cognitive  function  tests,  and  95%  for  the  3.2-km 
uphill  run.  Lor  all  parametric  statistical  comparisons,  p<0.01 
(Bonferroni  correction  of  0.05/5)  was  considered  significant,  with 
p<0.01  for  Wilcoxon  signed  rank  test  results  considered  signifi¬ 
cant.  Individual  data  for  all  responses  reported  here  are  presented 
in  Tables  SI,  S2,  S3,  S4,  S5.  Data  in  the  text  are  presented  as 
means  ±  standard  deviation. 

Results 

Anthropometry 

Height  and  body  mass  at  SL  are  presented  in  Table  1.  Body 
mass  was  unchanged  from  SL  to  ALT1  (p  =  NS),  then  dropped  by 
2.6±1.6kg  (p<0.01)  from  ALT1  to  ALT16;  it  showed  no 
significant  change  thereafter  (Table  SI). 

Arterial  Blood  Gases  and  Hemoglobin 

Pa02,  PaC02,  Sa02,  and  Ca02  were  reduced  with  acute 
exposure  to  5260  m  (SL  to  ALT1,  p<0.01;  Ligure  2,  panels  A-C, 
Table  S2),  while  pH  and  P50  increased  (p<0.01,  Ligure  2,  panels 
D  and  E)  and  [Hb]  was  unchanged  (p  =  NS,  Ligure  2,  panel  L). 
Pa02,  Sa02,  Ca02,  P50,  and  [Hb]  all  increased  from  ALT1  to 
ALT16,  while  PaC02  continued  to  fall  (p<0.01,  all  comparisons) 
and  pH  was  unchanged  (p  =  NS;  Ligure  2).  Sa02  at  POST7  was 
maintained  at  ALT  16  levels.  In  contrast,  Pa02,  Ca02,  P50,  and 
[Hb]  at  POST 7  decreased  from  ALT  16  (p<0.01)  and  approached 
ALT1  values.  PaC02  rose  at  POST7  from  ALT  16  values  and  was 
significantly  different  from  both  ALT1  and  ALT16  (p<0.01). 
Since  subjects  studied  at  POST21  had  incomplete  arterial  blood 
gas  data  at  all  time  points  but  SL;  those  data  are  qualitatively 
discussed,  but  data  in  the  text  and  figures  are  at  all  time  points  for 
the  POST7  group  only.  The  pattern  of  change  from  ALT  16  to 
POST21  was  similar  to  that  seen  from  ALT  16  to  POST7  for 
Pa02,  PaC02,  Sa02,  Ca02,  pH,  and  [Hb],  suggesting  possible 
retention  of  acclimatized  values  for  Sa02,  but  less  so  for  Pa02, 
PaC02,  Ca02,  P50,  pH,  and  [Hb]. 

Acute  Mountain  Sickness 

LLQ  and  AMS-C  were  highly  correlated  (R2  =  0.72,  p<0.001) 
and  identified  the  same  subjects  as  AMS  positive  at  ALT1;  for 
brevity,  only  the  LLQ  score  is  discussed  (see  Table  S3).  Eighty-one 
percent  (17/21)  of  subjects  had  AMS  (LLQ>3;  p<0.01  vs.  SL)  on 
the  evening  of  their  first  night  at  5260  m;  of  those  with  AMS 
nearly  half  had  severe  AMS  (LLQ>6;  p<0.01vs.  SL;  Ligure  3A). 
AMS  completely  resolved  in  all  subjects  as  acclimatization 
progressed  from  ALT1  to  ALT  16.  Upon  reascent  at  POST7 
subjects  remained  free  from  AMS.  On  POST21,  3/7  of  subjects 
again  developed  AMS  scores^  3  (p  =  NS  vs.  ALT  16),  but  none 
reported  severe  AMS.  Nobody  exhibited  HAPE  or  HACE. 

Cognitive  Function 

Repeat  tests  at  sea  level  pre-post  expedition  showed  no  major 
differences  between  individuals  or  group  values  (p>0.5)  and  were 
thus  averaged  to  provide  a  more  robust  SL  value  (Table  S4a-c). 
Live  of  nine  neurocognitive  tests  showed  marked  decrements  from 
SL  to  ALT1  (Simple  Reaction  Time-1,  Simple  Reaction  Time-2, 
Code  Substitution — Simultaneous,  Match  to  Sample  and  Proce¬ 
dural  Reaction  Time,  p<0.01,  Ligure  4);  no  change  from  SL  to 
ALT1  was  seen  for  Code  Substitution — Delayed  Recall,  Spatial 
Discrimination,  Go-No-Go,  and  Memory  Search  (p>0.05)  (Table 
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Figure  2.  Arterial  Blood  Gases  and  [Hb]  During  Acclimatization  and  Upon  Reascent.  Resting  indices  of  ventilatory  and  hematological 
acclimatization  at  SL,  ALT1,  ALT16,  and  POST7/21  demonstrating  acclimatization  after  16  days  at  a  constant  altitude  and  the  degree  of  retention  in 
these  variables.  ^Significantly  different  vs.  SL  (p<0.01);  ^  significantly  different  than  ALT1  (p<0.01);  ^significantly  different  than  ALT16  (p<0.01). 
doi:1 0.1 371/journal.pone.00921 91  .g002 


S4a-c).  Subsequent  analyses  focused  on  the  five  tests  that  showed  a 
change  with  acute  hypoxia.  Performance  improved  on  Simple 
Reaction  Time-1,  Simple  Reaction  Time-2,  Code  Substitution — 
Simultaneous,  Match  to  Sample,  and  Procedural  Reaction  Time 
as  acclimatization  progressed  from  ALT1  to  ALT16  (p<0.01, 
Figure  4).  At  POST7,  Code  Substitution — Simultaneous  and 
Match  to  Sample  showed  retention  of  acclimatization  compared  to 
ALT  16  (p<0.01,  Figure  4,  panels  C  and  D),  with  loss  of 
acclimatization  evident  for  Simple  Reaction  Time-2,  Procedural 
Reaction  Time  (p<0.01,  Figure  4,  panel  B  and  E),  and  a  trend  to 
loss  of  acclimatization  noted  for  Simple  Reaction  Time-1 
(p>0.01<0.05,  Figure  4,  panel  A).  No  cognitive  function  tests 
showed  retention  of  acclimatization  at  POST21. 


Exercise 

Laboratory  exercise  testing.  Peak  oxygen  uptake  at  SL  was 
3.4±0.8  1/min  and  fell  by  29 ±11%  to  2.3±0.6  1/min  at  ALT1 
(p<0.01),  with  no  change  observed  from  ALT1  to  ALT  16  (p  =  NS) 
(See  Table  S5).  Peak  power  output  at  SL  was  265 ±57  W;  it  fell  by 
34±7%  to  171  ±40  W  at  ALT1  (p<0.01),  and  like  peak  oxygen 
uptake,  it  did  not  improve  with  acclimatization.  Changes  in  resting 
arterial  oxygenation  and  [Fib]  from  SL  to  ALT1  to  ALT  16  were 
not  correlated  with  peak  oxygen  uptake  (p  =  NS). 

Field  exercise  testing.  Running  speed  was  44±5%  slower  at  ALT1 
compared  to  SL  (p<0.01;  Figure  5).  Running  speed  improved 
8 ±8%  from  ALT1  to  ALT  16  (p<0.01)  and  was  maintained  at 
POST7  (p  =  NS).  Subjects  maintained  acclimatized  (ALT  16) 
running  speed  at  POST7  despite  13%  lower  resting  [Hb]  and 
CaC>2.  After  21  days  at  low  altitude,  running  speed  tended  to  be 
slower  than  at  ALT  16  (p  =  0.06)  and  was  not  significantly  different 
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Figure  3.  Development  of  Acute  Mountain  Sickness,  Its 
Resolution  with  Acclimatization  And  Prevention  Upon  Reas¬ 
cent.  Percentage  of  subjects  reporting  moderate  to  severe  AMS  based 
on  LLQ  scores  of  >3,  or  >6,  respectively.  (A)  Symptoms  of  AMS  at  ALT1 
were  alleviated  at  ALT16  and  were  largely  absent  with  reascent  on 
POST7/21.  (B)  Mean  Pa02  and  median  LL  AMS  scores  reveal  no 
relationship  of  hypoxemia  to  AMS.  ^Significantly  different  than  SL 
(p<0.01);  ^significantly  different  than  ALT1  (p<0.01);  ^significantly 
different  than  ALT16  (p<0.01). 
doi:1 0.1 371/journal.pone.00921 91  .g003 

from  ALT1  (p  =  NS),  suggesting  a  partial  loss  of  acclimatization  in 
running  speed  by  POST21. 

Relationship  of  AMS,  Cognitive  Function  and  Exercise 
Performance  to  Arterial  Oxygenation  and  [Hb] 

During  acclimatization  AMS,  cognitive  function,  and  exercise 
performance  improved,  and  for  AMS  and  exercise  those 
improvements  were  retained  upon  reascent,  with  only  some  tests 
of  cognitive  function  showing  retention  of  acclimatization.  The 
changes  that  occurred  during  acclimatization  and  upon  reascent  in 
Pa02,  PaC02,  Sa02,  Ca02,  P50,  pH,  and  [Hb]  were  not  related 
on  an  individual  (all  correlations  r<0.5)  or  group  basis  (all 
comparisons  p>0.1)  to  AMS,  cognitive  function,  or  exercise 
responses.  However,  the  pattern  of  change  with  acclimatization  in 
Pa02,  PaC02,  Sa02,  Ca02,  P50,  pH  and  [Hb]  matches  the 
pattern  of  change  for  AMS,  cognitive  function,  and  exercise 
performance,  suggesting  an  underlying  but  complex  relationship 
between  oxygenation  and  other  aspects  of  acclimatization. 

Discussion 

In  this  paper,  we  have  presented  four  aspects  of  altitude 
acclimatization  through  a  16-day  initial  exposure  to  5260  m,  and 
upon  reascent  to  the  same  altitude  after  either  seven  or  2 1  days  at 


low  altitude.  We  found,  as  have  others  before  us  [20-30],  elevated 
arterial  oxygenation  and  [Hb],  resolution  of  symptoms  of  acute 
mountain  sickness  and  increased  exercise  performance  after  16 
days  residence  at  5260  m.  We  also  report  improvements  in 
measures  of  cognitive  performance  that  we  believe  represent  a 
novel  and  important  additional  indicator  of  acclimatization.  Most 
intriguing  was  finding  that  after  descending  to  low  altitude  for  one 
or  three  weeks,  physiological  evidence  of  acclimatization  persisted 
upon  returning  to  5260  m,  as  manifest  by  less  AMS,  retention  of 
improved  exercise  performance,  and  to  some  extent  cognitive 
performance. 

Physiology  of  Acclimatization 

The  elevations  in  arterial  oxygenation  and  [Hb]  from  ALT  1  to 
ALT  1 6  were  similar  to  those  measured  in  individuals  acclimatized 
for  at  least  10  days  at  altitudes  ranging  from  3800  m  to  5260  m 
[20,26,29,30].  For  example,  Lundby  et  al.  reported  that  [Hb]  and 
Ca02  increased  markedly  from  SL  to  two  weeks  at  4100  m,  but 
did  not  rise  further  at  eight  weeks  [26].  While  similar  data  do  not 
exist  for  the  rise  in  Pa02  and  fall  in  PaC02  with  ventilatory 
acclimatization  at  two  and  eight  weeks  at  a  fixed  high  altitude, 
Wagner  et  al.  reported  after  nine  weeks  at  5260  m  a  Pa02  of 
50±1  mmHg  and  a  PaC02  of  21  ±0.9,  values  similar  to  Pa02 
(45  ±3)  and  PaC02  (21  ±3)  in  the  present  study  after  16  days  at 
5260  m  [30].  Thus,  it  seems  that>14  days  at  4000  m  to  5000  m 
results  in  significant  acclimatization,  and  that  this  duration  of 
exposure  can  be  effective  to  test  acclimatization  and  its  subsequent 
retention  [30]. 

Sixteen  days  of  acclimatization  at  5260  m  was  effective  in 
reducing  the  incidence  of  AMS  from  81%  in  our  subjects  upon 
acute  exposure  to  0%  at  ALT  16,  a  finding  consistent  with  existing 
literature  [23,24,28].  These  findings  suggest  a  new  experimental 
approach  to  unraveling  the  pathophysiology  of  AMS.  To  our 
knowledge,  no  pathophysiological  studies  of  AMS  have  taken 
advantage  of  the  complete  protection  from  AMS  conferred  by 
acclimatization  by  comparing  individuals  upon  acute  ascent  to 
when  they  are  acclimatized,  or  upon  reascent  when  presumably 
the  factors  that  protect  from  AMS  will  stand  out  from  other  factors 
that  are  epiphenomena  to  the  acclimatization  process  but  not  key 
to  AMS  prevention. 

This  is  the  first  report  of  complete  recovery  of  cognitive  function 
to  sea  level  values  after  acclimatization  to  high  altitude,  supporting 
the  idea  that  cognitive  function  is  an  important  outcome  of 
acclimatization.  DANA  tests  have  negligible  practice  effects  (other 
than  spatial  discrimination,  which  asymptotes  after  the  second 
administration)  [17].  This  was  evident  in  the  current  study,  as  no 
significant  differences  were  detected  between  DANA  measures  on 
pre-  and  post-expedition  SL  tests.  We  found  that  the  five  tests 
showing  impairment  in  acute  hypoxia  all  returned  to  SL  values  by 
ALT16  (p<0.01,  Figure  4).  Barcroft  et  al.  reported  anecdotal 
impairment  in  cognitive  function  during  acclimatization,  but 
lacked  any  quantitative  evidence  [31].  Other  studies  have  reported 
effects  on  cognitive  function  in  acute  hypoxia  [32-36]  during 
experiments  and  expeditions  where  the  barometric  pressure  and 
environmental  conditions  were  different  at  each  testing  point,  such 
as  occurs  during  a  climbing  expedition  [37-39],  and  one  has 
speculated  about  the  recovery  of  cognitive  function  with  acclima¬ 
tization  [40] .  However,  none  of  those  studies  have  shown,  as  in  the 
present  study,  that  when  subjects  are  studied  at  the  same  altitude 
over  the  course  of  acclimatization  that  cognitive  function  improves 
to  sea  level  values.  DANA  tests  speed  and  accuracy  in  measures 
that  assess  attention,  simple  discrimination,  and  immediate  and 
incidental  memory.  Although  these  measures  offer  an  indication  of 
working  memory,  they  do  not  assess  complex  problem-solving  and 
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Figure  4.  Neurocognitive  Function  During  Acclimatization  and  Upon  Reascent.  Five  tests  of  cognitive  function  revealed  marked 
decrements  in  performance  from  SL  to  ALT1,  and  improvement  back  to  sea  level  values  by  ALT16.  Code  Substitution — Simultaneous  and  Match  to 
Sample  retained  levels  found  at  ALT16  on  POST7,  while  Simple  Reaction  Time-1,  Simple  Reaction  Time-2,  and  Procedural  Reaction  Time  essentially 
reflected  a  loss  of  during  acclimatization  upon  reascent  at  POST7.  None  of  the  cognitive  function  tests  showed  any  retention  of  acclimatization  at 
POST21.  (tp  =  throughput  =  mean  number  of  correct  responses  made  within  one  min).  ^Significantly  different  than  SL  (p<0.01);  Significantly 
different  vs.  ALT1  (p<0.01);  Significantly  different  vs.  ALT16  (p<0.01). 
doi:1 0.1 371/journal.pone.00921 91  .g004 


decision-making  aspects  of  executive  functioning,  which  may  be 
especially  relevant  for  people  working  at  high  altitudes.  Under¬ 
standing  the  mechanism  for  the  marked  resolution  of  the  initial 
decrement  in  cognitive  performance  that  occurs  in  acute  hypoxia 
has  potential  impact  [41]  for  anyone  visiting,  living,  or  working  at 
high  altitudes  where  impaired  cognitive  dysfunction  is  a  major 
challenge  [37,38,42]. 

Our  findings  for  submaximal  exercise  performance  are  consis¬ 
tent  with  other  reports  showing  improvements  during  acclimati¬ 
zation  [22,25,27,43]  with  no  change  in  peak  oxygen  consumption 
[2,22,26,44—51].  However,  in  retrospect,  we  question  the  practical 


Figure  5.  Field  Exercise  Testing  During  Acclimatization  and 
Upon  Reascent.  Uphill  running  speed  plotted  as  percent  change  from 
sea  level  improved  from  ALT1  to  ALT16  and  was  retained  at  POST7,  with 
a  trend  to  retention  at  POST21.  ^Significantly  different  vs.  SL  (p<0.01); 
^significantly  different  vs.  ALT1  (p<0.01). 
doi:1 0.1 371/journal.pone.00921 91  .g005 


relevance  of  these  all-out  efforts,  as  most  work  or  recreational 
activities  at  high  altitude  are  not  performed  to  exhaustion  or  as  fast 
as  possible.  For  example,  mountaineers  try  to  preserve  energy  to 
sustain  efforts  across  multiple  days  and  might  actually  put 
themselves  at  risk  of  serious  harm,  or  death,  if  they  truly  reached 
the  point  of  exhaustion.  Their  ability  to  cover  more  ground  faster 
while  preserving  a  functional  reserve  is  a  hallmark  of  acclimati¬ 
zation  supported  by  anecdotal  accounts  [52,53].  To  the  best  of  our 
knowledge,  only  one  study  before  the  present  report  has 
objectively  measured  this  type  of  submaximal  performance  [43]. 
The  physiology  behind  the  improvement  in  sustained,  self- 
regulated  submaximal  performance  at  altitude  remains  unex¬ 
plored  [2,22,26,43-52]. 

Physiological  Retention  of  Acclimatization:  Arterial  Blood 
Gases  and  Hemoglobin 

At  POST7/21,  Pa02  and  PaC02  values  ranged  between 
ALT  16  and  ALT1  values,  indicating  partial  retention  of  ventila¬ 
tory  acclimatization.  In  contrast,  Sa02  and  pH  remained  near 
ALT  16  acclimatized  levels  on  POST7/21.  We  calculated  a 
decreased  P50  from  ALT  16  to  POST7/21,  suggesting  a  left  shift 
in  the  oxyhemoglobin  dissociation  curve  upon  reascent  as  a 
possible  explanation  for  the  retention  of  acclimatized  values  for 
Sa02  at  POST7/21  [2].  These  findings  are  compatible  with  one 
previous  study  showing  partial  retention  of  ventilatory  acclimati¬ 
zation  using  noninvasive  indices  of  oxygenation  and  end  tidal  C02 
levels  after  eight  days  at  low  altitude  [2,54].  The  drop  in  [Hb] 
from  ALT  16  to  POST7/21  may  be  due  to  selective  destruction  of 
the  youngest  circulating  red  cells  (neocytolysis)  upon  return  to  low 
altitude  [55-58],  or  potentially  an  increase  in  plasma  volume  [59]. 
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Physiological  Retention  of  Acclimatization:  Acute 
Mountain  Sickness 

Our  findings  on  AMS  upon  reascent  extend  the  work  of  others 
conducted  at  lower  altitudes  in  demonstrating  that  previous 
altitude  acclimatization  confers  some  protection  from  AMS 
[3,4,60].  The  marked  efficacy  of  acclimatization  to  prevent  severe 
AMS  is  underscored  by  comparison  to  results  from  clinical  trials 
where  acetazolamide  only  reduced  the  risk  of  severe  AMS  by  44% 

[61] ,  compared  to  100%  for  acclimatization  in  our  study.  Exactly 
how  acclimatization  prevents  AMS  and  other  high-altitude 
illnesses  upon  reascent  is  unclear. 

AMS  is  clearly  triggered  by  hypoxemia,  but  once  the  processes 
that  cause  AMS  are  initiated,  the  relationship  with  Pa02,  Sa02, 
and  Ca02  is  less  clear.  This  is  reflected  in  Figure  3B  where  AMS 
scores  are  highest  when  Pa02  is  lowest  at  ALT1,  but  when  at 
POST7  and  ALT  16,  when  Pa02  levels  are  only  a  few  mmHg 
higher  than  ALT1  values,  AMS  is  absent.  Additionally,  at  POST7, 
when  AMS  is  absent  in  all  14  subjects,  Ca02  levels  are  much 
lower  than  at  ALT  16,  suggesting  a  limited  role  for  Ca02  in  the 
protection  from  AMS  observed  upon  reascent.  One  explanation 
may  be  that  the  absolute  value  of  Pa02,  Sa02,  or  Ca02  is  not  the 
critical  factor,  but  rather  that  acute  hypoxia  sets  in  motion  the 
physiological  alterations  leading  to  AMS.  In  other  words,  perhaps 
an  individual  threshold  exists  that  triggers  AMS  when  crossed 

[62] .  Unraveling  how  this  occurs  may  lead  to  advances  in  the 
understanding  of  the  pathophysiology  of  high-altitude  illnesses. 

Physiological  Retention  of  Acclimatization:  Cognitive 
Function 

Cognitive  function  stands  out  as  a  key  feature  of  acclimatization 
to  hypoxia  that  is  not  completely  retained  at  acclimatized  levels 
upon  reascent.  The  tests  that  showed  retention  of  acclimatization 
at  POST7  (Code  Substitution — Simultaneous  and  Match  to 
Sample)  commonly  reflect  changes  in  short-term  memory.  The 
tests  of  reaction  time  (Simple  Reaction  Time-1,  Simple  Reaction 
Time-2,  Procedural  Reaction  Time)  essentially  returned  to  ALT1 
values  by  POST7,  indicating  a  loss  of  the  improvement  in  reaction 
time  seen  with  acclimatization.  Short-term  memory  and  reaction 
time  appear  to  represent  distinct  processes  that  respond  differently 
to  the  changes  in  arterial  blood  gases  and  [Hb]  from  ALT  16  to 
POST7.  Understanding  the  mechanisms  responsible  for  acclima¬ 
tization  retention  or  its  loss  could  lead  to  new  insights  into  the  links 
between  brain  oxygenation  and  cognitive  function  for  persons  at 
high  altitudes. 

Physiological  Retention  of  Acclimatization:  Exercise 

The  retention  of  exercise  performance  for  at  least  seven  days, 
with  partial  retention  after  21  days  spent  at  low  altitude,  has 
important  implications  for  everyone  living,  visiting,  or  working  at 
high  altitudes.  At  POST7,  and  to  a  lesser  extent  at  POST21, 
subjects  essentially  matched  their  acclimatized  running  perfor¬ 
mance.  This  is  the  first  report  of  retention  of  acclimatized  exercise 
performance  upon  reascent  after  de-acclimatizing  at  low  altitude. 
As  far  as  we  know,  only  one  other  study  attempted  to  measure 
retention  of  acclimatized  endurance  exercise  performance  [2] ,  but 
that  study  showed  no  improvement  in  endurance  exercise 
performance  with  acclimatization,  likely  due  to  a  small  sample 
size  (n  =  6),  thus  rendering  testing  of  retention  impossible.  As  noted 
above,  all  studies  [22,25,27,43]  but  one  [2]  have  shown 
improvement  in  submaximal  endurance  capacity  with  acclimati¬ 
zation.  The  retention  of  exercise  performance  shown  at  POST 7 
occurred  despite  significant  reductions  in  resting  [Hb]  and  Ca02. 
These  findings  are  contrary  to  those  reporting  a  direct  positive 


effect  of  Ca02  on  exercise  performance  at  lower  altitudes  [25,63], 
but  agree  with  those  reporting  little  effect  of  Ca02  on  exercise 
performance  at  higher  altitudes  (>3500  m)  [64—66].  If  the 
improvement  of  exercise  performance  with  acclimatization  and 
its  retention  upon  reascent  is  not  directly  related  to  Ca02,  then 
other  factors  must  be  at  play.  One  possibility  is  that  mechanisms 
other  than  oxygen  delivery  could  boost  oxygen  transport  and  thus 
exercise  performance  during  acclimatization  and  upon  reascent, 
such  as  elevated  circulating  blood  levels  of  vasodilatory  substances 
(e.g.,  nitric  oxide  [67]  or  adenosine  [68])  or  other,  as  yet  unknown, 
processes.  Discovering  the  mechanisms  responsible  for  improving 
exercise  performance  with  acclimatization  and  its  retention  after 
acclimatization  has  potential  relevance  to  exercise  tolerance  in 
anyone  exposed  to  hypoxia. 

Physiological  Mechanisms  Explaining  Acclimatization  and 
its  Retention 

Acclimatization  transforms  a  lowlander  into  someone  who  is 
protected  from  high-altitude  illness,  has  improved  cognitive 
function,  and  has  better  exercise  performance  at  5260  m.  In  the 
present  study,  acclimatization-induced  improvements  in  AMS 
symptoms,  cognitive  function  and  exercise  performance  appear  to 
follow  the  time  course  of  ventilatory  and  hematological  acclima¬ 
tization.  But  after  extensive  analysis,  no  case  was  found  where  the 
degree  of  improvement  in  AMS  symptoms,  cognitive  function,  and 
exercise  performance  was  significantly  directly  correlated  to 
measured  indices  of  arterial  oxygenation  and  [Hb].  Further, 
arterial  oxygenation  and  [Hb]  were  poorly  correlated  with  the 
benefits  of  acclimatization  that  persisted  upon  reascent.  Though 
not  well  known,  Luft  et  al.  reported  on  the  retention  of 
acclimatization  based  on  studies  conducted  in  hypobaric  chambers 
on  climbers  returning  from  Nanga  Parbat  in  1938  [69].  The 
measurement  of  retention  was  tolerance  to  very  high  altitudes 
(>8000  m)  measured,  in  part,  by  deterioration  of  handwriting. 
They  noted  that  neither  the  hemoglobin  concentration  nor  the 
erythrocyte  count  were  responsible  for  the  persistence  of 
acclimatization.  While  we  acknowledge  the  inherent  limitations 
of  correlational  analyses,  the  disconnection  between  ventilatory 
and  hematological  acclimatization  and  physiological  function 
suggests  that  additional  mechanisms  are  involved  in  acclimatiza¬ 
tion  and  its  retention.  These  might  include  physiological  responses 
that  we  did  not  measure,  or  molecular  and  cellular  responses  in  a 
specific  tissue  such  as  brain  that  cannot  be  easily  measured  in 
humans.  In  subsequent  reports  we  will  pursue  a  linkage  between 
the  OMIGS  responses  and  the  physiological  adjustments  described 
here  to  explore  the  mechanisms  underlying  acclimatization  to  high 
altitude  and  its  retention. 

Limitations 

Several  limitations  in  the  study  design  and  execution  should  be 
considered.  This  study  was  completed  in  the  field,  in  a  foreign 
country,  and  with  many  uncontrolled  variables.  The  rationale  for 
this  approach  over  a  trial  in  a  hypobaric  environment  where  many 
more  variables  could  be  controlled  was  that  such  a  large  study 
could  not  be  completed  for  a  reasonable  cost  and  in  a  reasonable 
time-frame  in  a  hypobaric  chamber.  Operation  Everest  II  studied 
six-to-eight  subjects  during  a  40-day  simulated  ascent  of  Mt 
Everest.  Though  many  of  the  time  points  from  Operation  Everest 
II  had  data  from  only  four  to  six  subjects,  many  important 
observations  were  made  from  these  experiments  [29,44,70—73]. 
But  to  have  sufficient  statistical  power  to  combine  the  OMICS  and 
physiological  studies,  much  larger  sample  sizes  are  needed.  As  far 
as  the  authors  know,  there  is  one  hypobaric  chamber  in  the  world 
large  enough  to  accommodate  21  subjects  at  a  time,  located  in 
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Glasgow,  Scotland.  While  we  acknowledge  the  field  design  as  a 
limitation,  we  believe  this  study  makes  an  important  contribution 
to  understanding  acclimatization  that  can  point  to  future  studies 
with  smaller  samples  and  more  focused  experimental  questions  in 
controlled  hypobaric  chamber  conditions. 

This  study  was  limited  to  16  days  of  acclimatization.  While  this 
was  sufficient  time  to  see  marked  changes  in  the  variables 
measured,  it  is  unclear  if  longer  exposure  would  have  resulted  in 
further  improvements  in  acclimatization  or  better  retention  of 
acclimatization  upon  reascent.  Also,  due  to  logistical  and  financial 
constraints  and  to  avoid  areas  of  high  malarial  risk,  subjects  did 
not  descend  all  the  way  to  sea  level  between  exposures.  However, 
this  may  not  be  a  major  concern,  since  our  results  are  consistent 
with  other  studies  reporting  protection  from  AMS  after  acclima¬ 
tization  [3,4,60].  Only  Lyons  et  al.  [3]  reported  data  from  a 
controlled  study  of  acclimatizing  individuals;  others  used  epide¬ 
miological  observations  suggesting  AMS  protection  from  acclima¬ 
tization  [4,60].  Also,  we  made  no  measurements  at  low  altitude 
prior  to  reascent,  so  a  question  remains  as  to  how  much  of  the 
reascent  responses  were  present  at  low  altitude  such  as  hyperven¬ 
tilation,  resulting  in  low  PaC02,  versus  how  much  was  nascent  at 
low  altitude  but  was  rapidly  triggered  on  re-ascent. 

An  additional  concern  is  that  subjects  may  have  de-trained  over 
the  16  days  at  high  altitude,  since  they  were  unable  to  completely 
maintain  their  regular  exercise  regimen.  When  back  at  low 
altitude,  subjects  resumed  their  habitual  levels  of  physical  activity, 
potentially  restoring  some  fitness  and  confounding  our  measures  of 
exercise  performance.  Also,  changes  in  total  and  lean  body  mass 
across  the  study  may  have  affected  physical  performance  [74],  but 
since  changes  in  body  composition  and  training  status  are  inherent 
to  life  at  high  altitude,  we  feel  our  results  have  strong  practical 
relevance. 

Finally,  the  AltitudeOmics  project  encompasses  an  extensive 
suite  of  physiological  and  OMICS  measurements,  and,  in  its 
entirety,  produced  more  than  60  million  individual  data  points. 
Consequently,  the  data  has  been  partitioned  into  discrete  papers 
with  the  ultimate  goal  of  a  series  of  publications  that  are 
individually  robust  and  as  comprehensive  as  possible.  The 
physiological  parameters  included  in  this  paper  have  historically 
been  used  to  describe  acclimatization,  and  thus  were  deemed 
appropriate  as  a  bridge  between  past  studies  and  the  novel 
discoveries  from  AltitudeOmics.  Further  publications  will  explore 
the  process  of  acclimatization  by  utilizing  additional  OMICS  and 
physiological  data  whose  inclusion  excessively  widened  the  scope 
of  the  current  paper. 

Conclusion 

In  this  study  of  acclimatization  to  a  very  high  altitude,  we  found 
improvements  in  key  variables  after  16  days  that  describe  an 
acclimatized  phenotype  by  partial  acclimatization  for  arterial 
oxygenation  and  [Hb] ,  absence  of  high-altitude  illness,  improved 
cognition  and  exercise  performance.  Another  intriguing  observa¬ 
tion  is  that  after  descending  to  low  altitude  for  one  or  three  weeks, 
evidence  of  acclimatization  persists,  as  manifested  by  an  acclima¬ 
tized  value  for  Sa02,  much  less  severe  AMS,  maintained  exercise 
performance,  and  to  a  lesser  extent  retention  of  acclimatized 
cognitive  performance.  During  the  time  at  low  altitude,  many  of 
the  changes  reflecting  ventilatory  and  hematologic  adaptation 
returned  to  or  toward  the  unacclimatized  state  at  the  time  reascent 
measurements  were  made.  In  conclusion,  this  study  identifies  a 
phenotype  of  successful  human  acclimatization  to  hypoxia, 
identifies  novel  aspects  of  the  retention  of  the  acclimatized 
phenotype  after  time  at  low  altitude,  and  will  serve  as  a  foundation 
for  comparing  the  phenotype  of  acclimatization  with  potential 
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mechanistic  mediators  of  acclimatization  derived  from  companion 
studies  of  the  human  transcriptome,  epigenome,  metabolome,  and 
proteome. 
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AltitudeOmics:  Decreased  reaction  time  after  high  altitude 
cognitive  testing  is  a  sensitive  metric  of  hypoxic  impairment 

Emma  B.  Roach3,  Joseph  Bleibergb,  Corinna  E.  Lathan3,  Lawrence  Wolpert3, 
Jack  W.  Tsaoc  and  Robert  C.  Roachd 


Humans  experiencing  hypoxic  conditions  exhibit  multiple 
signs  of  cognitive  impairment,  and  high  altitude 
expeditions  may  be  undermined  by  abrupt  degradation 
in  mental  performance.  Therefore,  the  development 
of  psychometric  tools  to  quickly  and  accurately  assess 
cognitive  impairment  is  of  great  importance  in  aiding 
medical  decision-making  in  the  field,  particularly 
in  situations  where  symptoms  may  not  be  readily 
recognized.  The  present  study  used  the  Defense 
Automated  Neurobehavioral  Assessment  (DANA), 
a  ruggedized  and  portable  neurocognitive  assessment  tool, 
to  examine  cognitive  function  in  healthy  human  volunteers 
at  sea  level,  immediately  after  ascending  to  an  elevation 
over  5000  m,  and  following  1 6  days  of  acclimatization  to 
this  high  altitude.  The  DANA  battery  begins  with  a  simple 
reaction  time  test  (SRT1)  which  is  followed  by  a  20-min 
series  of  complex  cognitive  tests  and  ends  with  a  second 
test  of  simple  reaction  time  (SRT2).  Tabulating  the 
performance  scores  from  these  two  tests  allows  the 
calculation  of  an  SRT  change  score  (dSRT  =  SRT1-SRT2) 
that  reflects  the  potential  effect  of  mental  effort  spent 
during  the  20-min  testing  session.  We  found  that  dSRT, 

Introduction 

The  partial  pressure  of  oxygen  reduces  exponentially  with 
increasing  altitude  and  leads  to  hypoxia,  an  underlying 
cause  of  cognitive  and  physiological  impairment  at  high 
altitude.  In  general,  the  severity  of  impairment  is  a 
function  of  both  altitude  and  the  rate  of  ascent,  where 
moderate  altitudes  ( <  2000  m)  and  slow  elevation 
gains  induce  little  decrement  compared  with  extreme 
altitudes  (>  6000  m)  and  rapid  ascension,  which  have 
more  severe  effects  and  can  result  in  loss  of  consciousness 
or  death  [1]. 

Rapid  ascent  to  high  altitude  results  in  a  number  of 
impairments  in  cognitive  performance  (see  [2]  for  review), 
although  it  should  be  noted  that  fatigue  and  other  travel- 
related  factors  must  be  accounted  for  before  attributing 
impairment  to  hypoxia  alone  [3].  Impairments  due  to 
hypoxia  have  been  observed  in  short-term  memory  [4] ,  long¬ 
term  memory  and  verbal  expression  [5],  attention  [6,7],  and 
reaction  time  [8,9].  Because  of  the  potential  impact  of  these 
impairments  upon  high  altitude  expeditions,  the  develop- 
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but  not  direct  SRT  in  comparison  to  sea-level  baseline 
performance,  is  highly  sensitive  to  acute  altitude-related 
performance  deficits  and  the  remission  of  impairment 
following  successful  acclimatization.  Our  results 
suggest  that  dSRT  is  a  potentially  useful  analytical 
method  to  enhance  the  sensitivity  of  neurocognitive 
assessment.  NeuroReport  25:814-818  ©  2014  Wolters 
Kluwer  Health  |  Lippincott  Williams  &  Wilkins. 
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ment  of  field-deployable  tools  to  aid  the  assessment  of 
hypoxia-induced  cognitive  impairment  is  highly  relevant  to 
medical  decision-making  in  these  scenarios. 

In  this  report,  we  analyzed  an  unexamined  feature  of 
the  neurocognitive  data  collected  from  healthy  human 
volunteers  on  an  expedition  to  Mt  Ghacaltaya  in  Bolivia  [10]. 
Cognitive  performance  was  assessed  in  this  study  using  the 
Defense  Automated  Neurobehavioral  Assessment  (DANA), 
a  software  package  of  public  domain  cognitive  tests  that  runs 
on  the  Android  platform.  DANA  was  originally  developed  as 
a  means  of  rapidly  assessing  cognitive  changes  following  mild 
traumatic  brain  injury/concussion  in  deployed  service 
members  exposed  to  blasts,  and  its  reliability  has  been 
previously  validated  in  a  number  of  extreme  environments 
[11].  The  DANA  test  battery  includes  two  administrations 
of  a  simple  reaction  time  (SRT)  task:  one  at  the  beginning 
and  one  at  the  end  of  the  ~  20-min  test  session.  To 
investigate  the  hypothesis  that  the  second  measurement  of 
reaction  time  might  reveal  an  effect  of  mental  fatigue  on 
cognitive  performance,  we  tabulated  a  dSRT  score  to 
compare  throughput,  a  measure  of  cognitive  efficiency, 
between  the  two  reaction  time  administrations.  Here  we 
show  that  performance  decreases  across  DANA  testing  as  a 
function  of  acute  exposure  to  hypoxic  conditions,  an 
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altitude-related  decrement  that  resolves  following  successful 
physiological  acclimatization  to  high  altitude. 

Methods 

Volunteer  subjects 

As  part  of  the  AltitudeOmics  study  on  the  physiological 
signatures  of  altitude  acclimatization  [10],  DANA  was 
administered  to  a  group  of  volunteers  at  sea  level  (SL) 
and  at  5260  m  atop  Mt  Chacaltaya  near  La  Paz,  Bolivia. 
The  study  was  performed  according  to  the  Declaration  of 
Helsinki  and  was  approved  by  the  Institutional  Review 
Boards  of  the  University  of  Colorado  and  the  University  of 
Oregon,  as  well  as  the  Human  Research  Protection  Office 
of  the  US  Department  of  Defense.  The  detailed  methods 
for  the  overall  study  are  summarized  here  and  described 
elsewhere  [10].  Before  giving  written  and  verbal  consent 
to  participate,  each  volunteer  was  informed  of  the 
possible  risk  and  discomforts  involved  in  the  study.  From 
a  pool  of  79  volunteers,  a  total  of  24  were  recruited 
under  strict  criteria  including  birth  at  a  low  elevation 
(<  1500  m),  physical  fitness,  and  general  health  char¬ 
acteristics  (not  pregnant  or  lactating,  no  prescription  drug 
use,  and  no  history  of  migraine,  loss  of  consciousness, 
smoking,  cardiovascular  abnormality,  or  pulmonary  dys¬ 
function).  Of  the  recruited  participants,  three  dropped 
out  of  the  study  because  of  medical  reasons  apart  from 
altitude  sickness  (e.g.  gastrointestinal  illness),  resulting 
in  a  total  of  21  participants  (12  male,  nine  female;  mean 
age  20.8  years,  range  19-23  years).  The  constraints  of  the 
study,  including  strict  inclusion/exclusion  criteria,  travel 
costs,  and  subject  travel  availability,  produced  a  small  and 
relatively  homogenous  sample. 

The  experiment  proceeded  according  to  the  following 
timeline:  first,  the  participants  underwent  baseline 
testing  at  SL  (Eugene,  Oregon,  USA)  ~  1  month  before 
traveling  to  Bolivia.  After  an  overnight  flight  to  El  Alto 
(4050  m),  the  participants  immediately  descended  to 
Corocio  (1525  m)  where  they  rested  for  48  h  to  limit  the 
effects  of  jet  lag.  Next,  pairs  of  participants  were  driven  to 
the  top  of  Mt  Chacaltaya  (5260  m)  over  a  period  of  3  h. 
During  the  drive,  supplemental  oxygen  was  provided  to 
each  participant  through  either  a  mask  or  a  nasal  cannula 
(21/min)  to  allow  an  assessment  of  acute  change  upon 
reaching  the  destination  altitude.  After  the  ascent,  one 
member  of  each  pair  immediately  began  testing,  whereas 
the  other  continued  to  breathe  supplemental  oxygen  for 
2h  until  his/her  turn  for  testing.  The  assessment  was 
repeated  after  16  days  of  acclimatization  at  5260  m,  which 
included  descents  to  La  Paz  (3800  m)  over  the  first  4  days. 
A  final  round  of  sea-level  (SL)  testing  was  conducted  ~3 
months  after  returning  from  Bolivia  to  collect  data  from 
participants  who  missed  the  initial  SL  testing. 

DANA  administration 

The  DANA  test  battery  includes  the  following  tests: 
SRT1,  code  substitution  (simultaneous),  procedural 


reaction  time,  spatial  discrimination,  go/no-go,  code 
substitution  (delayed),  match  to  sample,  and  Sternberg 
memory  search.  The  ~20-min  test  battery  ended  with  a 
second  administration  of  simple  reaction  time  (SRT2) 
according  to  methodology  introduced  by  Bleiberg 
et  al.  [12].  The  test  battery  was  administered  on  a 
Trimble  Nomad  handheld  computer  (Android  version  2.1; 
Trimble,  Sunnyvale,  California,  USA).  Performance  in  the 
tests  administered  between  SRT1  and  SRT2  was 
analyzed  elsewhere  [10];  for  the  present  purposes,  the 
intervening  test  battery  between  the  two  SRT  adminis¬ 
trations  may  be  thought  of  as  providing  a  cognitive 
challenge  to  the  participant,  which  we  hypothesized  to 
have  a  negligible  effect  upon  SRT2  under  normal 
conditions. 

Each  SRT  administration  consisted  of  40  trials  with  a 
random  intertrial  interval  (600-3000 ms).  Each  trial 
began  when  a  yellow  target  appeared  on  a  black  screen. 
The  participant  was  instructed  to  tap  the  target  as  soon  as 
it  appeared  and  was  asked  to  perform  the  task  as  quickly 
and  accurately  as  possible.  The  participants  completed 
four  practice  trials  with  feedback  before  commencing  the 
portion  of  the  test  from  which  data  were  collected. 


Data  analysis 

Throughput  was  calculated  as  follows  for  each  SRT 
administration  per  participant: 


Throughput 


Correct  trials  ( % ) 

Correct  trial  median  reaction  time  (ms) 


x 60000 


If  the  percentage  of  incorrect  trials  (i.e.  failing  to  respond 
within  900  ms  or  responding  in  anticipation  of  the  cue) 
exceeded  33%  for  any  SRT  administration,  whether 
because  of  suboptimal  effort,  illness,  or  sleep  deprivation, 
the  participant  was  excluded  from  the  analysis  (n  —  2).  A 
two-way,  repeated  measures  analysis  of  variance  was  used 
to  analyze  global  effects  on  throughput.  Pairwise  compari¬ 
sons  were  examined  using  Bonferroni-corrected  paired 
/-tests  (significance  at  P  <  0.05/4  =  0.0125),  and  effect 
sizes  were  calculated  with  Cohen’s  d.  Because  the  average 
throughput  from  the  two  SL  administrations  was  not 
significantly  different  (unpaired  /-test,  P=0.51),  the 
second  administration  was  used  for  the  baseline  compari¬ 
son  as  it  included  complete  datasets  from  all  of  the 
participants.  All  analyses  were  carried  out  using  MATLAB 
R2013b  (Mathworks,  Natick,  Massachusetts,  USA). 


Results 

Participant  performance  in  a  simple  reaction  time  test  was 
assessed  at  the  beginning  (SRT1)  and  end  (SRT2)  of  a  20- 
min  DANA  testing  session  at  SL,  following  an  ascent  to 
5260  m  (ALT1),  and  after  16  days  of  acclimatization  to  this 
altitude  (ALT16;  Fig.  1).  A  two-way,  repeated  measures 
analysis  of  variance  with  the  factors  altitude  (SL,  ALT1,  or 
ALT16)  and  administration  (SRT1  or  SRT2)  revealed 
significant  main  effects  of  altitude  (F  =  15.96,  P  <  0.0001) 
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Fig.  1 


and  administration  (F  =  22.02,  P<  0.0005),  as  well  as  a 
significant  interaction  of  these  terms  (F  =  11.37,  P  <  0.0005) 
upon  SRT  throughput. 

Post-hoc  testing  detected  no  difference  between  SRT1 
and  SRT2  throughput  at  SL  (P=  0.43).  This  finding  was 
corroborated  by  additional  analysis  of  a  previously 
collected  dataset  [11],  in  which  a  similar,  ~15-min 
version  of  DANA  was  administered  to  groups  of  healthy 
volunteers  in  a  variety  of  extreme  climates  (desert, 
jungle,  aboard  a  ship,  and  at  high  altitude  postacclima¬ 
tization).  Paired  /-tests  comparing  throughput  in  SRT1 
versus  SRT2  failed  to  reach  significance  in  any  of  these 
climates  (all  P’s  >  0.05),  supporting  the  conclusion  that 
the  intervening  tests  in  the  DANA  battery  do  not 
adversely  impact  SRT  performance  in  healthy  humans 
under  these  conditions. 


Throughput  in  a  simple  reaction  time  (SRT)  task  as  a  measure  of 
cognitive  performance  during  acute  altitude  exposure,  after 
acclimatization,  and  at  sea  level  (SL).  SRT  was  administered  before 
(SRT1)  and  after  (SRT2)  cognitive  loading  with  DANA  testing. 
Following  acute  altitude  exposure  (ALT1),  both  SRT1  and  SRT2 
performance  decreased,  with  a  particular  decrement  in  the  second 
administration.  Following  two  weeks  of  acclimatization  (ALT16), 
performance  in  the  SRT  task  approximated  sea  level  (SL)  scores.  Error 
bars  represent  SE  from  the  mean.  DANA,  Defense  Automated 
Neurobehavioral  Assessment. 


Fig.  2 


(a) 

SRT  I -SRT  I  (SL) 


(b) 

dSRT  (SRTI-SRT2) 


Quantification  of  altitude  and  exertion-related  performance  changes, 
(a)  Simple  reaction  time  (SRT)  performance  during  acute  altitude 
exposure  (ALT1)  and  following  two  weeks  of  acclimatization  (ALT16) 
was  compared  with  sea-level  (SL)  performance.  There  was  no 
significant  difference  between  ALT1  and  ALT16  when  compared  with 
SL  throughput,  (b)  Change  in  SRT  performance  across  DANA  testing 
was  compared  for  each  time  point.  A  marked  decrease  in  throughput 
was  observed  at  ALT1,  whereas  ALT16  showed  an  indistinguishable 
difference  from  SL.  *P<0.005,  **P< 0.001.  DANA,  Defense 
Automated  Neurobehavioral  Assessment. 


To  quantify  the  apparent  altitude-induced  impairment  in 
reaction  time,  we  first  examined  SRT  performance  by 
comparing  against  baseline  values,  a  method  favored  by 
many  neurocognitive  assessment  protocols  [13,14].  Each 
participant’s  SL  ‘baseline’  SRT1  throughput  was  sub¬ 
tracted  from  the  SRT1  throughput  values  at  ALT1  and 
ALT16.  On  average,  participants  showed  a  throughput 
decrease  of  17.85  ±4.71/min  (mean±SE)  at  ALT1  and 
10.97±5.45/min  at  ALT16  compared  with  SL  baseline. 
A  paired  /-test  revealed  that  this  baseline  comparison 
measure  failed  to  show  a  significant  difference  between 
ALT1  and  ALT16  (P  =  0.24;  Fig.  2a). 

A  second  comparison,  dSRT,  was  calculated  as  the 
difference  in  each  participant’s  SRT1  and  SRT2 
throughput  scores  at  each  time  point  (SL,  ALT1,  and 
ALT16).  These  comparisons  revealed  that  SRT2  through¬ 
put  decreased  at  the  end  of  DANA  testing  by  an  average 
of  23.87±4.52/min  at  ALT1,  1.63±2.71/min  at  ALT16, 
and  5.71  ±3.42/min  at  SL,  showing  a  significant  differ¬ 
ence  between  ALT1  and  ALT16  (P  <  0.001)  and  between 
ALT1  and  SL  (P  <  0.005;  Fig.  2b).  In  addition,  the  dSRT 
comparisons  produced  much  larger  effect  sizes  than  the 
baseline  comparison  (dSRT  ALT1  vs.  ALT16  d—  0.95, 
ALT1  vs.  SL  d—  0.75;  baseline  d  =  0.28). 

Discussion 

In  accordance  with  previously  collected  evidence  from 
healthy  human  volunteers  [11],  no  difference  was 
detected  between  SRT1  and  SRT2  throughput  at  SL. 
These  results  support  the  hypothesis  that  DANA  testing 
does  not  induce  sufficient  cognitive  loading  to  alter 
psychomotor  performance  in  healthy  participants  under 
normal  inspired  oxygen  and  barometric  pressure.  How¬ 
ever,  a  comparison  of  performance  in  the  two  SRT 
administrations  unmasked  a  robust  altitude-dependent 
effect  of  cognitive  exertion  upon  psychomotor  efficiency. 
The  difference  score  dSRT  (SRT1  throughput  -  SRT2 
throughput)  shows  a  significant  relationship  with  acute 
altitude  exposure:  a  marked  decrease  in  throughput 
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following  cognitive  testing  emerges  after  ascent  from  SL. 
However,  following  16  days  of  acclimatization  to  high 
altitude,  throughput  scores  resemble  those  seen  at  SL: 
SRT1  and  SRT2  performances  are  indistinguishable.  These 
results  are  in  agreement  with  complementary  physiological 
and  cognitive  data  that  were  simultaneously  collected  from 
the  same  participants  [10].  In  contrast,  the  baseline 
comparison  measure  failed  to  show  a  significant  difference 
between  ALT1  and  ALT16.  These  results  indicate  that  in 
this  context,  comparison  against  baseline  was  not  sensitive 
to  the  cognitive  effects  of  acute  hypoxia  and  subsequent 
acclimatization.  Further,  the  dSRT  comparison  produced  a 
much  larger  effect  size  than  the  baseline  comparison, 
indicating  that  dSRT  is  a  robust  metric  by  which  cognitive 
impairment  may  be  quantitatively  assessed. 

A  similar  post-testing  decrease  in  reaction  time  was 
reported  by  Bleiberg  et  al.  [15]  in  a  study  on  fatigue  in 
postpolio  patients.  In  this  study,  an  Automated  Neuro¬ 
psychological  Assessment  Metrics  (ANAM)  battery  [12] 
was  used  with  a  configuration  similar  to  DANA:  an  SRT 
task  was  presented  both  at  the  beginning  and  at  the  end 
of  a  battery  of  more  complex  cognitive  tests.  The 
participants  began  the  morning  with  a  complete  ANAM 
battery,  underwent  a  1-h  comprehensive  functional 
medical  evaluation  including  motor  testing  and  other 
fatiguing  activities,  and  then  completed  a  second  round 
of  the  ANAM  battery.  Although  less  than  a  quarter  of  the 
postpolio  participants  showed  a  decrement  in  SRT1,  over 
50%  showed  decreased  performance  in  SRT2,  a  differ¬ 
ence  which  was  highly  statistically  significant.  Together 
with  the  present  results,  these  data  indicate  that 
performance  in  an  SRT  task  after  cognitive  loading  may 
be  a  highly  sensitive  means  for  observing  cognitive 
impairment.  However,  the  parameters  (e.g.  length, 
difficulty,  repetition,  etc.)  of  the  testing  battery  that 
are  responsible  for  the  observed  results  are  yet  to  be 
identified.  It  could  be  the  case  that  a  more  condensed 
assessment  may  be  sufficient  to  reveal  cognitive  impair¬ 
ment;  alternatively,  greater  sensitivity  to  impairment  may 
be  achievable  using  an  optimized  test  battery. 

Although  the  observed  decrement  in  SRT  performance 
upon  acute  hypoxia  exposure  could  be  interpreted  as 
motor  fatigue  rather  than  cognitive  impairment  per  se,  we 
note  that  several  of  the  other  reaction  time  tasks 
interleaved  within  the  test  battery  did  not  show  a 
significant  difference  between  SL  and  ALT1  [10].  Taken 
together,  these  results  indicate  that  decreased  motor 
output  alone  cannot  explain  the  change  in  performance; 
however,  more  research  is  required  to  investigate  the 
complex  interaction  of  cognitive  and  motor  processing 
under  these  conditions. 

Conclusion 

Comparing  SRT  performance  at  the  beginning  and  end 
of  a  DANA  test  battery  provides  a  more  robust  and 
reliable  indication  of  hypoxia-induced  cognitive  impairment 


than  the  typically  used  comparison  against  baseline  perfor¬ 
mance.  Because  SRT  throughput  does  not  decrease  across 
testing  under  normal  conditions,  these  results  suggest  that 
calculating  the  dSRT  score  is  a  promising  analytical  method 
that  may  aid  neurocognitive  assessment  in  situations  where 
appropriate  baseline  data  are  not  available. 

Acknowledgements 

This  paper  is  one  in  a  series  titled  AltitudeOmics’  that 
together  represents  a  group  of  studies  exploring  the  basic 
mechanisms  controlling  human  acclimatization  to  hypoxia 
and  its  subsequent  retention.  Many  people  and  organiza¬ 
tions  invested  enormous  time  and  resources  to  make  this 
project  a  success.  Foremost,  the  study  was  made  possible 
by  the  tireless  support,  generosity,  and  tenacity  of  our 
research  participants  (please  see  Subudhi  et  al.  [10]  for  a 
complete  list  of  people  and  organizations  who  contrib¬ 
uted  to  this  effort).  In  addition,  the  authors  would  like  to 
thank  Lindsay  Long  for  her  assistance  in  organizing  the 
data  and  James  Drane,  Julia  Kern,  and  Sonja  Jameson-Van 
Houten  for  their  assistance  with  data  collection.  Finally, 
the  authors  are  grateful  for  the  helpful  comments  and 
discussion  of  the  manuscript  by  James  Drane,  Clementina 
Russo,  and  James  Spira. 

This  study  was  funded  by  BUMED;  US  Department  of 
Defense  (W81XWH- 11-2-0040  TATRC);  NIH/NCATS 
Colorado  CTSI  (UL1  TR000154);  the  Cardiopulmonary 
&  Respiratory  Physiology  Laboratory,  University  of 
Oregon;  and  the  Altitude  Research  Center  and  the 
Charles  S.  Houston  Endowed  Professorship,  Department 
of  Emergency  Medicine,  School  of  Medicine,  University 
of  Colorado  Denver. 

Conflicts  of  interest 

Emma  B.  Roach,  Corinna  E.  Lathan,  and  Lawrence 
Wolpert  are  employed  by  AnthroTronix  Incorporated, 
developer  of  the  DANA  tool. 


References 

1  Petrassi  FA,  Hodkinson  PD,  Walters  PL,  Gaydos  SJ.  Hypoxic  hypoxia  at 
moderate  altitudes:  review  of  the  state  of  the  science.  Aviat  Space  Environ 
Med  2012;  83:975-984. 

2  Virues-Ortega  J,  Buela-Casal  G,  Garrido  E,  Alcazar  B.  Neuropsychological 
functioning  associated  with  high-altitude  exposure.  Neuropsychol  Rev 
2004;  14:197-224. 

3  Basnyat  B,  Cumbo  TA,  Edelman  R.  Acute  medical  problems  in  the  Himalayas 
outside  the  setting  of  altitude  sickness.  High  Alt  Med  Biol  2000;  1 : 
167-174. 

4  Bartholomew  CJ,  Jensen  W,  Petros  TV,  Ferraro  FR,  Fire  KM,  Biberdorf  D, 
et  al.  The  effect  of  moderate  levels  of  simulated  altitude  on  sustained 
cognitive  performance.  Int  J  Aviat  Psychol  1999;  9:351-359. 

5  Hornbein  TF,  Townes  BD,  Schoene  RB,  Sutton  JR,  Houston  CS.  The  cost  to 
the  central  nervous  system  of  climbing  to  extremely  high  altitude.  N  Engl  J 
Med  1989;  321:1714-1719. 

6  Stivalet  P,  Leifflen  D,  Poquin  D,  Savourey  G,  Launay  JC,  Barraud  PA,  et  al. 
Positive  expiratory  pressure  as  a  method  for  preventing  the  impairment  of 
attentional  processes  by  hypoxia.  Ergonomics  2000;  43:474-485. 

7  Bonnon  M,  Noel-Jorand  MC,  Therme  P.  Effects  of  different  stay  durations  on 
attentional  performance  during  two  mountain  expeditions.  Aviat  Space 
Environ  Med  2000;  71 :678-684. 


Copyright  ©  Lippincott  Williams  &  Wilkins.  Unauthorized  reproduction  of  this  article  is  prohibited. 


818 


NeuroReport  2014,  Vol  25  No  1 1 


8  Bolmont  B,  Thullier  F,  Abraini  JH.  Relationships  between  mood  states  and 
performances  in  reaction  time,  psychomotor  ability,  and  mental  efficiency 
during  a  31 -day  gradual  decompression  in  a  hypobaric  chamber  from  sea 
level  to  8848  m  equivalent  altitude.  Physiol  Behav  2000;  71:469-476. 

9  Sharma  VM,  Malhotra  MS,  Baskaran  AS.  Variations  in  psychomotor 
efficiency  during  prolonged  stay  at  high  altitude.  Ergonomics  1975; 
18:511-516. 

10  Subudhi  A,  Bucher  J,  Bourdillon  N,  Davis  C,  Elliott  J,  Eutermoster  M,  et  al. 
AltitudeOmics:  the  integrative  physiology  of  human  acclimatization  to 
hypobaric  hypoxia  and  its  memory  on  reascent.  PLoS  One  201 4;  9:e921 91 . 

1 1  Lathan  C,  Spira  JL,  Bleiberg  J,  Vice  J,  Tsao  JW.  Defense  Automated 
Neurobehavioral  Assessment  (DANA)-psychometric  properties  of  a  new 
field-deployable  neurocognitive  assessment  tool.  Mil  Med  2013;  178:365-371 . 


12  Bleiberg  J,  Cernich  AN,  Cameron  K,  Sun  W,  Peck  K,  Ecklund  PJ,  et  al. 
Duration  of  cognitive  impairment  after  sports  concussion.  Neurosurgery 
2004;  54:1073-1078,  Discussion  1078-1080. 

13  Echemendia  RJ,  Iverson  GL,  McCrea  M,  Macciocchi  SN,  Gioia  GA 
Putukian  M,  et  al.  Advances  in  neuropsychological  assessment  of 
sport-related  concussion.  Br  J  Sports  Med  2013;  47:294-298. 

14  Guskiewicz  KM,  Bruce  SL,  Cantu  RC,  Ferrara  MS,  Kelly  JP,  McCrea  M,  etal. 
National  Athletic  Trainers’  Association  Position  Statement:  management  of 
sport-related  concussion.  J  Athl  Train  2004;  39:280-297. 

1 5  Bleiberg  J,  Johnson  D,  Maxwell  S,  Kenney  K,  Campbell  W,  Vasconcelos  O. 
Computerized  assessment  of  cognitive  fatigue  in  survivors  of  paralytic 
poliomyelitis.  132rd  Annual  Meeting  of  American  Neurological  Association, 
2007,  Washington,  DC. 


Copyright  ©  Lippincott  Williams  &  Wilkins.  Unauthorized  reproduction  of  this  article  is  prohibited. 


Appendix  D 

JOURNAL  OF  NEUROTRAUMA  31:1823-1834  (November  15,  2014) 

©  Mary  Ann  Liebert,  Inc. 

DOI:  1 0.1 089/neu.201 4.3363 


The  Impact  of  Multiple  Concussions  on  Emotional  Distress, 
Post-Concussive  Symptoms,  and  Neurocognitive  Functioning 
in  Active  Duty  United  States  Marines  Independent 
of  Combat  Exposure  or  Emotional  Distress 


James  L.  Spira,1  Corinna  E.  Lathan,2  Joseph  Bleiberg,3  and  Jack  W.  Tsao4 


Abstract 

Controversy  exists  as  to  whether  the  lingering  effects  of  concussion  on  emotional,  physical,  and  cognitive  symptoms  is 
because  of  the  effects  of  brain  trauma  or  purely  to  emotional  factors  such  as  post-traumatic  stress  disorder  or  depression. 
This  study  examines  the  independent  effects  of  concussion  on  persistent  symptoms.  The  Defense  Automated  Neurobe- 
havioral  Assessment,  a  clinical  decision  support  tool,  was  used  to  assess  neurobehavioral  functioning  in  646  United  States 
Marines,  all  of  whom  were  fit  for  duty.  Marines  were  assessed  for  concussion  history,  post-concussive  symptoms, 
emotional  distress,  neurocognitive  functioning,  and  deployment  history.  Results  showed  that  a  recent  concussion  or  ever 
having  experienced  a  concussion  was  associated  with  an  increase  in  emotional  distress,  but  not  with  persistent  post- 
concussive  symptoms  (PPCS)  or  neurocognitive  functioning.  Having  had  multiple  lifetime  concussions,  however,  was 
associated  with  greater  emotional  distress,  PPCS,  and  reduced  neurocognitive  functioning  that  needs  attention  and  rapid 
discrimination,  but  not  for  memory-based  tasks.  These  results  are  independent  of  deployment  history,  combat  exposure, 
and  symptoms  of  post-traumatic  stress  disorder  and  depression.  Results  supported  earlier  findings  that  a  previous  con¬ 
cussion  is  not  generally  associated  with  post-concussive  symptoms  independent  of  covariates.  In  contrast  with  other 
studies  that  failed  to  find  a  unique  contribution  for  concussion  to  PPCS,  however,  evidence  of  recent  and  multiple 
concussion  was  seen  across  a  range  of  emotional  distress,  post-concussive  symptoms,  and  neurocognitive  functioning  in 
this  study  population.  Results  are  discussed  in  terms  of  implications  for  assessing  concussion  on  return  from  combat. 

Key  words:  adult  brain  injury;  behavioral  assessments;  cognitive  function;  head  trauma;  military  injury 


Introduction 

Approximately  1.5  million  Americans  survive  a  traumatic 
brain  injury  (TBI)  from  traffic  accidents,  assaults,  sports,  and 
work  injuries,  with  the  vast  majority  of  these  being  primarily  mild 
(mTBI),  otherwise  known  as  concussion.1  Concussion,  however,  is 
uniquely  problematic  in  the  military  given  the  new  strategies  of  war 
encountered  by  service  members  when  fighting  an  insurgency  using 
improvised  explosive  devices.  The  rate  of  concussion  experienced 
by  United  States  (U.S.)  service  members  engaging  in  combat 
during  the  wars  in  Afghanistan  and  Iraq  has  been  estimated  at 
between  15%  and  22%. 2-4 

Concussion  can  affect  cognitive,  somatic,  and  emotional  func¬ 
tioning,  including  slowed  and  less  accurate  information  processing, 
sensory  sensitivity,  imbalance,  headaches,  and  emotional  lability.5 


Although  most  symptoms  of  concussion  appear  to  dissipate  within 
hours  or  days,  lingering  effects  of  concussion  have  been  widely 
reported,6-8  with  persistent  post-concussive  symptoms  (PPCS)  re¬ 
ported  in  between  15%  and  50%  of  persons  with  mTBI,  depending 
on  how  it  is  defined  and  assessed.5  Several  studies  show  that  at  least 
a  subgroup  of  persons  do  continue  to  have  both  subjective  symp¬ 
toms,9-13  physical  symptoms,14  and  lower  cognitive  test  perfor¬ 
mance,  even  after  90  days.15,16 

It  has  been  argued,  however,  that  the  rates  of  PPCS  are  actually 
much  lower  or  that  any  lingering  symptoms  are  not  from  concussion 
at  all,  with  any  persistent  effects  on  cognitive  and  somatic  func¬ 
tioning  exclusively  because  of  the  presence  of  concomitant  emo¬ 
tional  problems  such  as  post-traumatic  stress  disorder  (PTSD)  and 
depression.2,17-21  Indeed,  the  extent  to  which  the  effects  of  concus¬ 
sion  persist  has  been  a  topic  of  intense  debate  with  substantial 
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clinical  and  policy  implications,  including  recommendations  that  the 
military  halt  post-deployment  assessment  of  post-concussive  diffi¬ 
culties,  focusing  instead  on  emotional  difficulties  associated  with 
cognitive  and  somatic  complaints.22 

The  discrepancies  between  reports  of  PPCS  in  the  literature  may 
arise  from  several  factors.  First,  the  populations  investigated  may 
vary.  Rates  and  symptom  manifestations  of  PPCS  across  studies 
may  be  because  of  differential  levels  of  emotional  comorbidity  in 
patients  who  were  concussed  from  sports  injury  compared  with 
accident,  assault,  or  combat.  Findings  may  also  vary  based  on 
whether  the  sample  studied  was  assessed  in  a  clinical  setting, 
leading  to  the  possibility  of  exaggeration  of  self-reported  symp¬ 
toms  because  of  secondary  gain,  compared  with  an  anonymous 
sample  outside  a  clinic  setting. 

Second,  the  influence  of  number  and  recency  of  concussions  may 
not  have  been  adequately  assessed.  Most  studies  compare  the  pres¬ 
ence  of  a  previous  or  recent  concussion  versus  no  concussion.  The 
number  of  lifetime  concussions,  however,  has  not  been  adequately 
evaluated.  A  few  recent  studies  have  begun  to  show  an  association  of 
multiple  concussions  with  persistent  neurocognitive  and  emotional 
symptoms.  This  is  seen  in  long-term  follow-up  studies  of  profes¬ 
sional  contact  sports23-25  as  well  as  among  active  duty  personnel.26 
For  instance,  Bryan  and  Clemens27  recently  found  an  association 
between  lifetime  number  of  concussions  and  suicidal  ideation  in  a 
small  sample  of  recently  concussed  military  personnel,  independent 
of  the  effects  of  depression  and  PTSD,  yet  whether  these  symptoms 
persisted  weeks  after  the  injury  was  not  assessed. 

Thirdly,  the  degree  of  co-occurring  emotional,  somatic,  and 
cognitive  disorders  may  cloud  the  assessment  of  the  individual 
contributions  of  these  factors  to  overall  functioning,  requiring 
sufficient  assessment  of  these  components  along  with  analytic 
methods  to  tease  apart  their  individual  effects.28  Because  there  is  a 
large  overlap  in  post-concussive  symptoms  with  symptoms  com¬ 
mon  in  patents  with  depression  and  PTSD,29-31  it  is  difficult  to 
distinguish  among  these  unless  assessments  of  both  concussion  and 
emotional  distress  are  administered  together.  It  is  additionally 
complex  to  assess  the  unique  contribution  of  concussion  to  func¬ 
tioning  in  combat  veterans  because  of  the  stress  of  combat  during 
which  service  members  are  dealing  with  the  ever  present  threat  of 
death,  being  away  from  loved  ones,  and  the  stress  of  deployment. 
Indeed,  Cooper  and  associates17  found  that  deployment  stress  was 
associated  with  PPCS,  and  Vesterling  and  colleagues32  found 
cognitive  functioning  declining  across  the  deployment  cycle,  in¬ 
dependent  of  concussion. 

It  is  because  of  such  an  association  between  emotional  distress 
commonly  seen  in  combat  and  post-concussive  complaints  that 
many  argue  that  post-concussive  complaints  are  the  direct  result  of 
emotional  distress  alone.  It  has  also  been  argued,  however,  that 
concussion  lowers  the  threshold  for  developing  emotional  prob¬ 
lems.33  Morrisette  and  coworkers34  found  evidence  suggesting  that 
recently  deployed  service  members’  concussions  led  to  higher  rates 
of  depression  and  PTSD  after  deployment;  Maguen  and  col¬ 
leagues35  found  an  association  between  multiple  concussions  and 
rates  of  PTSD  2  years  later  in  combat  veterans;  and  Bryan26  found 
an  association  between  number  of  concussions  and  sleep  difficulty 
in  deployed  service  members.  Despite  the  many  comorbid  condi¬ 
tions  that  exist  in  military  members  who  have  been  concussed, 
distinguishing  the  unique  contributions  of  concussion  from  the 
effects  of  emotional  distress  is  not  always  considered  when 
studying  the  causes  of  PPCS. 

Finally,  few  studies  examining  PPCS  have  assessed  both  ob¬ 
jective  measures  of  functioning,  such  as  cognitive  performance, 


along  with  subjective  self-report  of  emotional  and  somatic  func¬ 
tioning.  It  is  well  known  that  cognitive  functioning  can  be  impaired 
for  at  least  the  first  few  days  after  concussion.5,36  Speed  and  ac¬ 
curacy  (and  their  combined  measure  of  mental  efficiency, 
throughput  [TP])  have  been  shown  to  decline  across  a  range  of 
cognitive  functions,  including  simple  reaction  time37  as  well  as 
delayed  memory  and  executive  functioning.15  Although  this 
problem  has  been  discussed  for  decades,  few  studies  have  exam¬ 
ined  the  long-term  effects  of  concussion  on  post-concussive  neu¬ 
rocognitive  functioning  and  corresponding  subjective  reports  of 
cognitive,  sensory,  somatic,  and  emotional  symptoms.38  The  in¬ 
terpretation  of  cognitive  dysfunction  after  concussion  is  compli¬ 
cated,  because  it  is  also  known  that  emotional  distress  affects 
cognitive  functioning.  This  may  be  even  more  complicated  in 
combat  veterans,  because  several  studies  have  also  shown  that 
cognitive  performance  is  affected  by  the  stress  of  having  been 
combat  deployed. 32,39,40 

In  light  of  the  above  discussion,  there  is  little  consensus  as  to  the 
nature  of  persistent  post-concussive  symptoms  in  those  with  a 
previous  concussion.  Therefore,  the  aim  of  this  study  was  to  de¬ 
termine  whether  the  recency  and  number  of  lifetime  concussions 
had  a  sustained  impact  on  emotional,  somatic,  and  cognitive 
functioning  in  a  sample  of  U.S.  Marines,  independent  of  deploy¬ 
ment  stress,  and  symptoms  of  depression  and  PTSD.  A  cohort  of 
U.S.  Marines  was  thought  to  be  ideal  for  this  examination  because 
of  the  uniformity  of  the  sample,  the  high  rates  of  exposure  to  blast, 
the  high  rates  of  co-occurring  PTSD  and  depression,  and  the  high 
level  of  functioning  compared  with  a  patient  population,  allowing 
for  an  examination  of  the  effects  of  concussion  recency  and  number 
on  cognitive  and  somatic  functioning,  relatively  independent  of 
self-report  emotional  difficulties  or  exaggeration  of  symptom  be¬ 
cause  of  secondary  gain  compared  with  a  clinic  sample.  In  addition, 
Marines  from  the  same  unit  who  had  not  been  deployed  or  who  had 
been  previously  (but  not  recently)  deployed  allowed  for  an  analysis 
of  the  independent  effects  of  deployment  stress.  Determining  the 
separable  effects  of  these  factors  will  aid  in  the  debate  over  whether 
assessment  and  treatment  of  concussion  should  continue  in  the 
military. 

Methods 

Participants 

U.S.  Marines  (n  =  646)  from  the  Second  Expeditionary  Force 
(2MEF)  participated  in  the  single  testing  session  reported  here. 
Volunteer  participants  were  all  active  duty  service  members  in  good 
standing  (not  on  limited  duty  medical  status  or  pending  medical  or 
disciplinary  discharge).  Testing  took  place  in  2012  at  Marine  Base 
Camp  Lejeune,  where  an  entire  neurobehavioral  battery  was  ad¬ 
ministered,  and  at  Marine  Corp  Air  Grand  Combat  Center,  Twenty- 
nine  Palms,  where  the  entire  neurobehavioral  battery  was  admin¬ 
istered  to  some  participants,  but  because  of  time  constraints,  only  a 
portion  of  the  neurobehavioral  battery  was  administered  to  others 
(see  list  of  measures).  Marines  were  recruited  for  participation  in¬ 
dependent  of  their  medical  or  deployment  history. 

Procedure 

Medical  officers  arranged  for  2  hours  of  time  off  from  duties  for 
service  members  who  might  be  interested  in  participating  in  this 
study,  informing  them  that  researchers  were  developing  an  instru¬ 
ment  that  might  eventually  be  used  to  improve  medical  problems  in 
the  field.  Service  members  showed  up  at  an  appointed  time  in  a 
classroom  to  hear  a  briefing  about  the  research.  After  an  explana¬ 
tion  of  the  procedure  that  took  place  without  the  presence  of 
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commanding  officers  or  senior  enlisted  personnel  in  their  chain  of 
command,  volunteers  were  allowed  to  leave  or  sign  the  Department 
of  Defense  Institutional  Review  Board  approved  consent  to  par¬ 
ticipate.  No  remuneration  was  offered,  and  care  was  taken  to  insure 
that  participation  was  voluntary,  without  recrimination  for  refusing 
to  participate  or  leaving  during  testing.  No  identifiable  information 
was  collected,  other  than  age,  rank,  and  sex.  There  was  greater  than 
a  99%  agreement  to  participate  from  those  who  came  to  hear  the 
briefing. 

Groups  of  approximately  20-40  consented  service  members 
were  administered  neurocognitive  and  self-report  and  psychologi¬ 
cal  measures  on  a  handheld  computing  device,  in  classrooms  at 
either  Marine  Base  Camp  Lejeune  or  Marine  Desert  Warfare 
Training  Center,  Twentynine  Palms.  The  full  (Standard)  testing 
battery  reported  here  took  approximately  40  min  to  complete,  with 
the  subset  (Rapid  or  Brief)  battery  taking  approximately  10  min 
(see  list  of  measures).  After  an  explanation  of  the  procedure, 
handheld  devices  were  distributed.  For  the  neurocognitive  tests, 
participants  were  instructed  to  answer  as  rapidly  and  accurately  as 
possible.  All  tests  were  self-explanatory,  beginning  with  written 
instructions.  After  completion  of  the  test  battery,  service  members 
were  administered  additional  testing  for  determining  reliability. 
Only  the  first  test  administrations  are  discussed  here,  because  re¬ 
liability  of  this  instrument  has  been  reported  elsewhere.41 

Materials 

Testing  materials  was  administered  on  a  handheld  computing 
device,  programmed  with  the  Defense  Automated  Neurobehavioral 
Assessment  software  (DANA).  DANA  combines  a  traditional  Neuro 
Cognitive  Assessment  Tool  (NCAT)  with  other  behavioral  assess¬ 
ments.  All  subjects  were  administered  the  DANA  Standard  battery 
consisting  of  eight  different  neurocognitive  tests  assessing  speed  and 
accuracy  across  a  range  of  simple  motor  and  visual  discrimination  to 
more  complex  memory  and  decision-making  tasks,  as  well  as  five 
self-report  psychological  assessments.  Subjects  used  a  Trimble  NO¬ 
MAD  handheld  computing  device  and  were  instructed  to  use  a  stylus 
for  consistency.  The  NOMAD  uses  the  Android  Operating  System 
and  has  a  3.5-inch  color  pressure  sensitive  touch  screen.  This  format 
is  optimally  suited  for  tests  of  speed  and  accuracy  because  it  has  less 
variability  than  can  be  achieved  across  various  desktop  computers. 

DANA  was  funded  by  and  developed  for  the  Department  of 
Defense  for  use  by  medical  support  personnel  in  the  field  to  detect 
impaired  performance  because  of  deployment-related  factors  in¬ 
cluding  concussion,  emotional  distress,  or  exhaustion.  Although 
DANA  uses  standard  assessments  tools  that  have  been  well  re¬ 
searched  and  widely  utilized,  because  of  its  unique  delivery  system, 
DANA  was  tested  under  extreme  field  conditions  as  well  as 
classroom  settings,  and  found  to  have  adequate  test-retest  (>0.70) 
and  split-half  (>0.80)  reliability41  and  cross-platform  validity 
compared  with  the  Automated  Neuropsychological  Assessment 
Metric  (ANAM),  the  standard  neurocognitive  assessment  used  by 
the  military. 42Neurocognitive  tests  in  the  DANA  Standard  Battery 
used  to  assess  speed  and  accuracy  included  Simple  Reaction  Time 
(SRT1),  Procedural  Reaction  Time  (PRT),  Go-No-Go  (GNG), 
Spatial  Rotation  Discrimination  (SPD),  Code  Substitution  Si¬ 
multaneous  (CDS),  Code  Substitution  Recall  (CDR),  Sternberg 
Memory  Test  (STN),  and  a  second  Simple  Reaction  Time  (SRT2). 

Depending  on  the  test,  between  20  and  40  trials  per  test  are 
administered,  and  average  speed,  accuracy,  and  TP  are  calculated 
for  each  of  these  tests.  The  Standard  Battery  also  included  five  self- 
report  psychological  tests:  Post-Traumatic  Symptom  Disorder 
Check  List,  military  version  (PCL-M),  Primary  Care  Health 


Questionnaire  depression  module,  minus  the  suicide  question 
(PHQ-8);  Pittsburgh  Sleep  Quality  Inventory  (PSQI),  the  Deploy¬ 
ment  Stress  Inventory  (DSI),  a  modification  of  the  Neurobehavioral 
Symptom  Inventory  adapted  for  deployed  service  members  that 
includes  questions  about  anger  and  distress,  and  the  Combat  Ex¬ 
posure  Survey  (CES).  For  each  behavioral  assessment,  total  score 
was  calculated.  Accepted  clinical  threshold  scores  were  calculated 
for  dichotomous  analyses.  Demographic  and  concussion  history 
was  also  included  in  the  battery. 

Neurocognitive  measures.  Data  Cleaning  was  conducted 
for  neurocognitive  tests.  Responses  that  are  considered  too  fast 
(faster  than  150  msec  for  SRT  and  250  for  other  measures)  and 
lapses  (slower  than  850  msec  for  SRT  or  timed-out  for  the  response 
time  for  other  tests,  approximately  2000  msec)  are  considered  in¬ 
correct  responses  and  not  calculated  in  mean,  Standard  deviation 
(SD),  percent  correct,  or  TP  scores.  Neurocognitive  tests  that  have 
less  than  66%  correct  responses  are  considered  invalid.  Valid 
summary  test  scores  were  calculated  to  determine  mean  correct 
(Mean),  SD,  percent  correct  (%  correct),  and  TP  (mean  correct 
responses  divided  by  total  trials  within  1  min),  a  test  of  cognitive 
efficiency.  Descriptions  of  all  assessment  measures  are  listed 
below. 

SRT.a,b,c.  The  subject  taps  on  the  location  of  the  yellow  as¬ 
terisk  symbol  as  quickly  as  possible  each  time  it  appears.  This  task 
measures  pure  reaction  time. 

PRTa,b,c.  The  screen  displays  one  of  four  numbers  for  3  sec. 
The  subject  presses  on  a  left  button  (“2”  or  “3”)  or  right  button 
(“3”  or  “4”)  depending  on  the  number  pressed.  A  choice  reaction 
time  measure  of  accuracy,  reaction  time,  and  impulsivity.  This 
choice  reaction  time  task  targets  simple  executive  functioning  with 
easy  decision-making  capabilities. 

GNGa,b,c.  This  is  a  forced  choice  reaction-time  task  relevant 
to  warfighters.  A  house  is  presented  on  the  screen  with  several 
windows.  Either  a  “friend”  (green)  or  “foe”  (white)  appears  in  a 
window.  The  respondent  must  push  a  “fire”  button  only  when  a 
“foe”  appears.  A  choice  reaction  time  measure  of  sustained  at¬ 
tention  and  impulsivity.  The  test  assesses  speed  and  accuracy  of 
targets,  omissions,  and  commissions. 

SPDb,c.  Pairs  of  four-bar  histograms  are  displayed  on  the 
screen  simultaneously,  and  the  subject  is  requested  to  determine 
whether  they  are  identical.  One  histogram  is  always  rotated  ei¬ 
ther  ±90  degrees  with  respect  to  the  other  histogram.  Assesses  vi- 
suospatial  analytic  ability. 

CDS.C.  Subjects  refer  to  a  code  set  of  nine  symbol-digit  pairs 
that  are  shown  across  the  upper  portion  of  the  screen.  A  sequence  of 
single  symbol-digit  pairs  is  shown  below  the  key,  and  the  subject 
indicates  whether  or  not  the  single  pair  matches  the  code  by 
pressing  Yes  or  No.  Assesses  visual  scanning  and  attention, 
learning,  and  immediate  recall. 

Code  Substitution-Delayed  Memory b,c  (CDD).  After  a 
delay  of  several  intervening  tests,  the  same  symbol-digit  pairs  are 
presented  without  the  code.  The  subject  indicates  whether  or  not  the 
pairing  was  included  in  the  code  that  was  presented  in  the  earlier 
Code  Substitution  learning  section.  Assesses  learning  and  short¬ 
term  memory. 


lDANA  Rapid  Battery. 
’DANA  Brief  Battery. 
’DANA  Standard  Battery. 
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STN.C.  The  subject  memorizes  a  set  of  five  letters,  after  which 
letters  appear  on  the  screen  one  at  a  time,  and  the  subject  deter¬ 
mines  if  the  letter  on  the  screen  is  a  member  of  the  memory  set. 
Assesses  working  memory. 

Self-report  measures.  Self-report  psychological  question¬ 
naires  were  programmed  in  DANA  and  administered  on  the  handheld 
device  after  the  neurocognitive  assessments.  One  question  appeared 
at  a  time.  Wording  was  unchanged  from  the  original  questionnaires. 
Service  members  were  instructed  to  answer  as  accurately  as  possible, 
but  that  they  could  skip  a  question  if  it  disturbed  them. 

PCL-M.C.  A  17-item  scale  assessing  symptoms  in  response  to 
stressful  military  experiences.  This  scale  assesses  PTSD,  with 
subscales  including  Re-Experiencing,  Avoidance/Numbing,  and 
Hyperarousal.  Higher  scores  indicate  increased  PTSD  symptom¬ 
atology.  In  a  military  population,  scores  >  50  are  in  persons  likely 
to  have  PTSD.  For  greatest  specifically,  scores  >44  with  three  Re- 
Experiencing,  one  Avoidance/Numbing,  and  two  Hyperarousal 
endorsed  as  at  least  “most  of  the  time”  are  more  specific  for  PTSD 
and  correlate  very  highly  (0.92)  with  the  Clinician  Administered 
PTSD  Scale  (CAPS).  3 

Primary  Care  PTSD  Screen10  (PC-PTSD).  Four  screening 
questions  designed  for  use  in  clinical  settings  to  screen  for  PTSD, 
with  three  of  four  endorsed  items  suggestive  of  likely  PTSD. 
Questions  assess  Hyperarousal,  Re-Experiencing,  and  Avoidance 
for  PTSD  Screening.  This  test  is  more  sensitive  than  specific,  but 
correlates  highly  with  the  PCL.44,45 

PHQ8.  An  eight-item  Depression  Scale  assessing  symptom 
severity  and  diagnostic  criteria  for  major  depressive  disorder.  For 
research  purposes,  item  #9  (concerning  suicide)  was  not  included, 
yet  previously  published  results  indicate  that  the  scoring,  reliabil¬ 
ity,  and  clinical  validity  are  similar.46  A  person  with  a  score  of  0-9 
is  likely  to  have  no  depression,  10-14  mild  depression,  15-19 
moderate  depression,  and  20  +  severe  depression. 

CES.C.  A  seven-item  self-report  measure  that  assesses  war¬ 
time  stressors  experienced  by  service  members.  The  total  CES 
score  (ranging  0  to  41)  is  calculated  by  using  a  sum  of  weighted 
scores,  which  can  be  classified  into  one  of  five  categories  of  combat 
exposure  ranging  from  “light”  to  “heavy.”  This  scale  rates  cu¬ 
mulative  combat  exposure  and  is  highly  predictive  of  PTSD,  pain 
and  injury,  TBI,  depression,  and  other  behavioral  sequellae.47 

DSI.  The  DSI  is  an  adaptation  of  the  Neurobehavioral 
Symptom  Inventory-22  (NSI),  with  six  additional  questions  as¬ 
sessing  anger  and  distress  adapted  for  a  deployment  context. 
Consistent  with  others,48  factor  analysis  revealed  factors  referred  to 
as  post-concussive  symptom  subscales  in  the  current  analysis: 
Neurocognitive  (worsening  focus,  concentration,  memory),  Sen¬ 
sory  (alterations  in  sensitivity  to  light,  sound,  smell),  Somatic 
(increased  pain,  sleep,  discomfort),  and  Vestibular  (difficulties  with 
balance  or  feeling  dizzy),  as  well  as  total  score.  Because  emotional 
complaints  are  better  assessed  through  other  measures  included  in 
the  battery,  they  are  not  included  in  the  current  analysis  of  the  DSI. 

Insomnia  Screening  Index13  (I SI).  A  five-item  scale  evalu¬ 
ating  perceived  insomnia  severity  and  sleep  habits.  Each  item  is 
rated  on  a  five-point  scale  (0-4).  The  total  score  ranges  0  to  28,  and 
higher  scores  indicate  more  severe  insomnia.  A  cutoff  score  of  10 
has  been  shown  to  indicate  insomnia.49 

Pittsburgh  Sleep  Quality  Index0  (PSQI).  Nineteen  self- 
rated  items  and  5  partner-rated  items,  which  measure  sleep  quality 


during  the  previous  month.  This  scale  differentiates  “good”  from 
“poor”  sleepers  based  on  seven  areas:  subjective  sleep  quality, 
sleep  latency,  sleep  duration,  habitual  sleep  efficiency,  sleep  dis¬ 
turbances,  use  of  sleeping  medication,  and  daytime  dysfunction 
over  the  last  month.  This  scale  is  the  most  widely  used  sensitive  and 
specific  self-report  measure  for  insomnia.  A  score  above  6  indicates 
a  “poor”  sleeper.50 

Demographics  and  background  questionnaire.  Service 
members  were  asked  about  age,  gender,  rank,  deployment  history. 

Concussion  questions.  Based  on  the  Post-deployment 
Health  Assessment  used  for  all  deployed  personnel,  service  mem¬ 
bers  were  asked  about  their  history  of  concussion,  including 
number  of  lifetime  concussions,  when  their  most  recent  concussion 
occurred,  the  immediate  effects  of  the  concussion  on  their  cognitive 
status  (dazed  and  confused,  loss  of  memory,  loss  of  consciousness), 
and  current  post-concussive  symptoms. 

Hypotheses 

The  relationship  of  recency  and  number  of  lifetime  concussions 
on  emotional  distress,  post-concussive  symptoms,  and  neurocog¬ 
nitive  functioning  was  examined.  It  was  predicted  that  recency  of 
last  concussion  and  number  of  lifetime  concussion  would  be  neg¬ 
atively  associated  with  emotional  distress,  post-concussive  symp¬ 
toms,  and  neurocognitive  functioning,  and  further  that  concussion 
uniquely  contributes  to  these  factors  independent  of  PTSD  (PCL-M) 
or  depression  (PHQ-8)  scores  or  deployment  factors  (time  since 
deployment,  CES). 

Planned  analyses 

IBM  SPSS  v20.0  was  used  to  calculate  power  and  conduct  all 
analyses.  Sample  size  was  initially  determined  from  a  power  esti¬ 
mate  of  the  differences  between  two  groups  (concussed  -  not 
concussed)  to  find  at  least  a  small  effect  size  in  SRT  or  PCL,  based 
on  the  expected  presence  of  at  least  15%  reported  concussion,  15% 
PCL-M  scores  >  50,  and  15%  PHQ  >  10  in  those  recently  returned 
from  a  recent  combat  deployment.  Power  was  also  calculated  based 
on  stability  of  measures  and  expected  standard  errors  determined  in 
the  Phase  1  pilot  study.41 

Descriptive  reporting  includes  mean  correct  responses  (Mean), 
SD,  percent  correct  (%Correct),  and  TP  for  all  neurocognitive 
scores;  mean  and  SD  for  self-report  psychological  scores,  and  mean 
or  number  of  cases  and  SD  for  age,  sex,  and  number  and  recency  of 
concussions.  TP  was  used  for  analyses  of  interest  for  all  cognitive 
tests,  except  for  GNG  and  CDD  where  %Correct  is  the  metric  of 
relevance. 

The  effects  of  recency  of  concussion  and  number  of  lifetime 
concussions  on  neurocognitive,  emotional,  and  post-concussive 
scores  are  examined  first  with  a  univariate  General  Linear  Model  on 
continuous  variables.  Generalized  Linear  Modeling  (with  Wald 
statistic  and  Robust  Estimator)  was  used  to  confirm  significance 
when  groups  were  associated  with  unequal  variances  per  the  Levine 
test  of  homogeneity.  Pairwise  comparisons  between  type  of  de¬ 
ployment  and  number  of  concussions  using  the  Bonferroni  cor¬ 
rection  were  used  to  adjust  for  type-I  error.  PTSD  (PCL-M), 
depression  (PHQ-8),  deployment  history  (never,  recently  within  6 
months,  and  previously — longer  than  6  months  ago),  and  combat 
exposure  (CES)  scores  were  used  as  covariates  to  determine  the 
unique  contribution  of  concussion  to  emotional,  post-concussive, 
and  neurocognitive  measures.  PCL-M  and  PHQ-8  scores  were  not 
used  as  covariates  when  they  were  the  outcome  under  investigation. 

Logistic  Regression  was  used  to  determine  the  odds  ratios  (OR) 
for  the  contribution  of  concussion  to  clinical  cutoff  scores  for 
emotional  measures  (PCL-M  >49;  PHQ-8  >10;  DSI- Anger  >1 
SD),  and  for  neurocognitive  functioning  scores  more  than  one  SD 
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from  the  mean.  PCL-M,  PHQ-8,  deployment  history,  and  CES 
covariates  are  included  in  these  analyses  as  well  unless  the  cov¬ 
ariate  is  the  outcome  measure  of  interest  (PCL-M,  PHQ-8).  Effect 
sizes  are  reported  as  partial  eta2  (rj2p)  for  analysis  of  variance  and 
ORs  for  Logistic  Regression.  Description  of  effect  sizes  as  small, 
medium,  and  large  is  based  on  the  Cohen  interpretation.51  Because 
this  is  a  single  period  analysis,  data  will  be  analyzed  for  the  exact 
number  of  subjects  who  took  the  tests  in  the  analysis  without  ex¬ 
trapolating  for  missing  data. 

Results 

The  total  sample  size  consisted  of  646  U.S.  Marines  with  an 
average  age  of  22.7  years  (SD  =  2.7),  including  18  females.  Be¬ 
cause  the  chain  of  command  was  asked  not  to  be  present,  almost  all 
service  members  were  junior  enlisted  (E2-E4).  There  were  234 
service  members  who  had  never  been  deployed,  98  who  were 
previously  deployed  (more  than  1  year  before  assessment)  and  314, 
all  men,  who  were  recently  deployed  (returning  from  combat  de¬ 
ployment  in  Afghanistan  within  3-6  months  of  testing).  Of  the  646 
service  members  who  were  administered  the  assessment  tool,  369 
received  the  Standard  Battery  and  227  were  administered  the  Rapid 
or  Brief  Battery.  Because  the  Rapid  and  Brief  Batteries  are  con¬ 
tained  within  the  Standard  Battery,  calculations  are  based  on 
ft  =  419  involving  measures  of  PCL-M,  PSQI,  DSI,  SPD,  CDS, 
CDD,  and  STN,  and  646  for  all  other  measures  (CES,  PHQ-8,  SRT, 
PRT,  GNG). 

Of  the  entire  sample,  25%reported  having  been  previously 
concussed  at  some  point  in  their  lifetime  at  least  once,  7%  reported 
two  previous  concussions,  and  9%  reported  three  or  more  times. 
Deployment  status  (never,  previous,  recent)  and  CES  scores  were 
associated  with  number  and  recency  of  concussions,  PCL-M,  PHQ-8; 
DSI-Anger,  DSI-Total,  PSQI,  and  the  majority  of  the  neurocognitive 
tests  with  moderate  to  large  effect  sizes.  Therefore,  deployment  status 
and  CES  were  included  as  covariates  in  all  analyses.  PCL-M  and 
PHQ-8  are  also  included  whenever  they  were  not  the  outcome  vari¬ 
ables  of  interest. 

Effects  of  concussion  on  emotional  symptoms 

Despite  including  covariates,  having  ever  had  one  or  more 
concussions  was  associated  with  PCL-M  scores  (F=  15.55, 
p<  0.000,  >/2p  =  0.034),  PHQ-8  scores  (F  =  5.38,  p<  0.021, 
rj2 p  =  0.021),  and  DSI-Anger  (F=9.67,  p  <  0.002,  r/2p  =  0.023) 
compared  with  never  having  received  a  concussion.  This  was  likely 
because  of  the  presence  of  multiple  concussions,  however,  because 
having  had  only  one  concussion  was  not  associated  with  emotional 
distress  greater  than  having  no  concussions.  Recency  of  concussion 
was  significant  for  PCL-M  scores  (F  =  9.93,  p  <  0.000,  rj2 p  =  0.046); 
PHQ-8  scores  (F  =  6.67,  p<  0.001,  rj2p  =  0.031);  and  DSI-Anger 
(F=10.0,  /?<  0.000;  rj2 p  =  0.055).  Multiple  concussions  was  sig¬ 
nificant  for  PCL-M  (F  =  7.62,  pc.OOOO,  rj2p  =  0.053),  PHQ-8 
(F  =  4.54,  p<  0.004,  rj2 p  =  0.032),  and  DSI-Anger  (F  =  5.79, 
p<  0.001,  rj2 p  =  0.041). 

Bonferoni  corrected  pairwise  comparisons  showed  that  a  recent 
concussion  was  associated  with  more  distress  than  distant  con¬ 
cussion,  and  three  or  more  lifetime  concussions  were  associated 
with  more  distress  than  none  or  one  concussion  (at  least  p  <  0.05  in 
all  cases).  Two  lifetime  concussions  were  not  significantly  different 
from  zero  or  one,  or  from  three  concussions. 

No  interaction  between  recency  and  number  of  concussions  was 
found  for  either  PHQ-8,  PCL-M,  or  DSI-Anger,  although  those  who 
reported  having  three  or  more  lifetime  concussions  with  the  most 


recent  within  the  past  6  months  averaged  a  notably  higher  score 
(PHQ=11;  PCL  =  44;  DSI-Anger  =  5.5)  compared  with  those 
without  a  history  of  concussion  (PHQ  =  5;  PCL  =  31;  DSI- 
Anger  =  1.5).  The  lack  of  statistical  significance  for  these  analyses 
was  likely  because  of  low  sample  sizes  for  those  who  were  recently 
concussed  and  had  multiple  concussions;  power  was  far  too  low  to 
detect  a  significant  difference  (observed  power  was  between  0. 10  to 
0.16). 

Binary  Logistic  Regression  with  forward  stepwise  Wald  model 
to  eliminate  shared  variance  and  including  covariates  was  signifi¬ 
cant  for  three  versus  zero  or  one  lifetime  concussions  predicting 
PCL  >50  (Wald  =  22.11,  p  <0.000,  OR  =  2.48,  confidence  interval 
[Cl]:  1 .70-3.63);  PHQ  >10  (Wald  =  4.88, /? < 0.027,  OR  =  2.19,  Cl: 
1.09-4.40),  and  DSI-Anger  >1  SD  (Wald=  10.15,  /?<0.001, 
OR  =  4.36,  Cl:  1.76-10.80). 

Thus,  the  presence  of  at  least  one  earlier  concussion,  indepen¬ 
dent  of  deployment  and  CES,  was  associated  with  emotional  dis¬ 
tress,  with  small  effect  sizes  (Table  1).  Multiple  and  recent 
concussions,  independent  of  covariates,  were  associated  with 
emotional  distress  with  stronger  effect  sizes,  and  three  or  more 
lifetime  concussions  more  than  doubled  the  odds  of  reaching 
clinical  cutoff  scores  for  PTSD  and  depression,  and  had  four  times 
the  odds  of  anger  problems. 

Effects  of  concussion  on  post-concussive  symptoms 

To  explore  the  relationship  of  concussion  to  reporting  of  per¬ 
sistent  post-concussive  symptoms,  continuous  variables  of  the  DSI 
subscales  and  sleep  issues  from  the  PSQI  were  analyzed  incorporat¬ 
ing  covariates  of  previous  deployment,  CES,  PCL-M,  and  PHQ-8. 
The  presence  of  at  least  one  previous  concussion  was  not  related 
to  DSI-Cognitive  symptoms,  DSI-Sensory  symptoms,  DSI- Ves¬ 
tibular,  or  DSI-Total,  but  was  associated  with  DSI-Somatic 
symptoms  with  a  very  small  effect  size  (F  =  5.67,  p<  0.018, 
rj2 p  =  0.014),  and  with  sleep  problems  on  the  PSQI  (F=  13.93, 
p<  0.000,  t]2 p  =  0.038).  Planned  comparisons  of  number  of  con¬ 
cussions  using  the  Bonferroni  correction,  however,  showed  that 
DSI-Somatic  and  insomnia  findings  were  likely  because  of  the 
presence  of  multiple  concussions,  because  only  one  previous  con¬ 
cussion  did  not  significantly  vary  from  no  history  of  concussions. 

Recency  of  concussion  was  associated  with  DSI-Sensory 
(F= 5.56, p< 0.004,  i/2p  =  0.026),  DSI-Somatic  (F= 3.00,/) <0.002, 
rj2 p  =  0.024),  DSI- Vestibular  (F= 6.45,  p<  0.002,  t/2p  =  0.030),  and 
PSQI  (F  =  6.5 1 ,  p  <  0.002,  ;/2p  =  0.036),  but  not  with  DSI-Cognitive 
or  DSI-Total.  Number  of  concussions  was  associated  with  DSI- 
Somatic  (F  =  3.01,  p<  0.027,  rj2  p  =  0.022),  DSI- Vestibular 
(F  =  2.67,  /?<  0.047;  rj2p  =  0.019),  and  sleep  disturbance  (PSQI, 
F  =  5.42,  p<  0.001,  rj2 p  =  0.044),  but  not  with  DSI-Cognitive,  DSI- 
Sensory,  or  DSI-Total.  Three  or  more  concussions  compared  with 
none  or  one  previous  concussion  was  associated  with  DSI-Sensory 
(F  =  8.56,  p  <  0.004,  rj2p  =  0.022),  DSI-Somatic  (F=  8.75,  p  <  0.023, 
f/2p  =  .023),  and  DSI- Vestibular  (F  =  8.65,  p<  0.003,  r\2p  =  0.022), 
but  not  for  DSI-Cognitive,  DSI-Total  or  insomnia.  Two  previous 
concussions  were  not  significantly  different  from  zero  or  one 
concussion  in  any  post-concussive  symptom  outcome,  except  for 
insomnia  (/?<  0.004)  (Table  2). 

A  two-way  ANCOVA  with  independent  variables  Recency  of 
Concussion  and  Number  of  Concussions,  including  Previous  De¬ 
ployment,  CES,  PCL-M,  and  PHQ-8  as  covariates,  found  significant 
or  marginally  significant  interactions  for  DSI-Cognitive  (F=4.25, 
df= 2,406,/)  <0.021,  )/2p= 0.021),  DSI-Sensory  (F=2.64,  df=  2,406, 
p< 0.072,  ;/2p  =  0.0 13).  DSI-Somatic  (F=2.95,  df=2,406,  p  =  0.054, 
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Table  1.  Influence  of  Concussion  History  on  Emotional  Tests 


Emotional  tests  (with  all  covariates  included ) 


PCL-M 

PHQ-8 

DSI-Anger 

N  Mean  (SD)  p  value  (ES) 

Mean  (SD)  p  value  (ES) 

Mean  (SD)  p  value  (ES) 

Any  past  concussion: 

No 

300 

27.7  (11.8) 

4.8  (4.9) 

1.6  (2.2) 

Yes 

116 

34.2  (15.4) 

0.000(m) 

6.3  (5.3) 

0.021(s)* 

2.6  (2.2) 

0.002(s)* 

Never 

305 

27.5  (11.7) 

4.7  (4.9) 

1.46  (2.1) 

>6  mos.  ago 

98 

33.6  (14.8) 

6.0  (5.1) 

2.27  (2.4) 

<  6  mos.  ago 

15 

42.1  (17.6) 

0.000(m) 

9.9  (5.5) 

0.001(s)* 

4.31  (3.3) 

0.000  (m)* 

0 

305 

27.5  (11.7) 

4.7  (4.9) 

1.6  (2.2) 

1 

36 

32.2  (13.4) 

5.7  (4.0) 

2.0  (2.5) 

2 

33 

31.2  (14.6) 

5.3  (5.1) 

2.4  (2.9) 

3  + 

43 

39.4  (16.9) 

0.000  (m) 

8.3  (5.4) 

0.004(s)* 

3.5  (2.7) 

0.001(m) 

0-1  vs.  3+  > 

387 

0.000  OR  =  2.48 

0.027  OR  =  2. 19 

0.001  OR  =  4.36 

[criteria] 

[PCL-M  >50] 

[PHQ-8  >10] 

[DSI-Anger  >  1SD] 

PCL-M,  Post-traumatic  Symptom  Disorder  Check  List,  military  version;  PHQ-8  Primary  Care  Health  Questionnaire;  DSI-Anger,  Deployment  Stress 
Inventory;  SD,  standard  deviation;  ES  ,  effect  size;  OR,  odds  ratio. 

*Significant  (p<  0.001)  with  a  moderate  to  large  effect  size  before  including  covariates  of  previous  deployment,  CES,  PCL-M,  and  PHQ-8. 

The  influence  of  any  previous  concussion,  recency  of  most  recent  concussion  (less  than  or  greater  than  6  months  since  the  most  recent  concussion), 
number  of  Lifetime  Concussions  (0,  1,  2,  3,  or  more)  and  three  or  more  versus  zero  or  one  concussions  on  emotional  and  post-concussive  symptom 
reporting,  showing  mean  and  standard  deviation  (SD).  General  Linear  Model  (ANCOVA)  results  for  significance  level  ( p  value)  and  effect  size  (ES- 
small  [s],  medium  [m],  or  large  [1])  using  partial  eta-squared  effect  size  (t]2 p)  are  based  on  the  throughput,  with  Combat  Exposure  Survey,  Deployment, 
PCL-M,  and  PHQ-8  serving  as  covariates  (except  when  the  covariate  is  itself  the  dependent  variable).  Bivariate  Logistic  Regression  reports  Wald  for 
unique  contributions  and  odds  ratio  for  three  or  more  versus  zero  or  one  lifetime  concussions. 


t/2p=0.014),  DSI-Total  (F=3.02,  df=2,406;  p<0.05,  »?2P  =  0.015), 
but  not  DSI- Vestibular  or  PSQI,  with  those  who  had  three  or  more 
concussions  with  the  most  recent  concussion  within  6  months  re¬ 
porting  more  distress  than  those  who  had  none  or  one  concussion  in 
the  more  distant  past.  ES  were  small  for  all  analyses,  likely  because 
of  insufficient  power. 

Binary  Logistic  Forward  Stepwise  Wald  Regression  was  used  to 
determine  the  influence  of  three  or  more  concussions  compared 
with  zero  or  one  concussion  on  DSI  scales  being  one  SD  above  the 


mean.  Results  showed  three  or  more  concussions  predicting  DSI- 
Sensory  (Wald  =  7.42,  p  =  0.006,  OR=1.81,  Cl:  1.15-2.76)  and 
DSI- Vestibular  (Wald  =  4.24,  p  =  0.04,  OR  =  2.57,  Cl:  1.05-6:74), 
but  not  DSI-Cognitive,  DSI-Somatic,  DSI-Total,  or  PSQI  when  all 
covariates  were  included.  Without  covariates,  all  DSI  subscales 
were  significant  with  at  least  moderate  ES. 

In  general,  independent  of  deployment,  CES,  PCL-M,  and  PHQ-8 
scores,  the  presence  of  a  previous  concussion  was  not  related  to 
post-concussive  reporting,  with  the  exception  of  more  insomnia. 


Table  2.  Influence  of  Concussion  History  on  Post-Concussive  Tests 


Post-concussive  tests  (with  all  covariates  included) 


DSI-Cognitive  DSI-Sensory  DSI-Somatic  DSI-Vestibular  DSI-Total  PSQ-I 


N 

M(SD) 

P  (ES) 

M(SD) 

P  (ES) 

M(SD) 

p  (ES)  M(SD) 

P  (ES) 

M(SD) 

P  (ES) 

N 

M(SD) 

P  (ES) 

Any  past  concussion: 

No 

300 

.38(.47) 

.13(.25) 

.36(.33) 

.15(31) 

37(36) 

256 

53(3.2) 

Yes 

116 

.57(.55) 

n.s.* 

.25(.40) 

n.s.* 

.55(.44) 

.018(s)*  .30(.47) 

n.s.* 

.56(.46) 

n.s.* 

99 

7.8(4. 1) 

.000(m) 

Never 

305 

.38(.46) 

.12(.25) 

.36(33) 

.15(30) 

34(35) 

257 

53(3.2) 

>  6  mos.  ago 

137 

.56(.52) 

.21(.35) 

.52(.41) 

.25(.43) 

.51  (.42) 

84 

7.6(4. 1) 

<6  mos.  ago 

16 

.76(.70) 

n.s.* 

.53(.57) 

.004(s)* 

.86(.37) 

.007(s)*  .59(36) 

.002(s)* 

.82(33) 

n.s.* 

15 

8. 7(3. 9) 

.002(m) 

0 

305 

.38(.46) 

.12(.25) 

.36(.33) 

.15(30) 

37(37) 

257 

53(3.2) 

1 

36 

.48(.47) 

.19(.30) 

.46(.39) 

.21(38) 

.47(.40) 

28 

7.4(.4.0) 

2 

33 

.57(.55) 

.18(.35) 

.50(.44) 

.21(37) 

.53(.46) 

29 

7. 8(4.0) 

3  + 

43 

.69(.58) 

n.s.* 

.37(.47) 

n.s.* 

.71(.43) 

.027(s)*  .45(37) 

.047(s)* 

.70(.48) 

n.s.* 

43 

8.0(.4.2) 

.001  (m) 

0-1 

341 

.38(.46) 

.13(.25) 

.37(.33) 

.15(31) 

38(37) 

285 

53(3.3) 

Vs. 3  + 

43 

.71(.49) 

n.s.* 

.38(.47) 

.004(s)* 

.71(.43) 

.003(s)*  .46(37) 

.003(s)* 

.71(.40) 

n.s.* 

43 

8.0(43) 

n.s.* 

DSI,  Deployment  Stress  Inventory  ;  PSQI,  Pittsburg  Sleep  Quality  Inventory  ;  SD,  standard  deviation;  ES,  effect  size. 

^Significant  (p<  0.001)  with  a  moderate  to  large  effect  size  before  including  covariates  of  previous  deployment,  Combat  Exposure  Survey,  Post- 
Traumatic  Symptom  Disorder  Check  List,  military,  and  Primary  Care  Health  Questionnaire- 8. 

The  influence  of  any  previous  concussion,  recency  of  most  recent  concussion  (less  than  or  greater  than  6  months  since  the  most  recent  concussion), 
number  of  Lifetime  Concussions  (0,  1,  2,  3,  or  more),  and  three  or  more  versus  zero  or  one  concussions  on  emotional  and  post-concussive  symptom 
reporting,  showing  mean  (M),  standard  deviation  (SD),  p  value  (p),  and  effect  size  (ES-  small  [s],  medium  [m],  or  large  [1]). 


IMPACT  OF  MULTIPLE  CONCUSSIONS  ON  SYMPTOMS 
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Recency  of  concussion,  number  of  concussions  (especially  having 
had  three  or  more  lifetime  concussions),  as  well  as  the  interaction 
between  number  and  recency  of  concussions,  however,  was  asso¬ 
ciated  with  worse  post-concussive  symptoms  reporting.  ES  were 
large  when  covariates  were  not  included,  suggesting  the  synergistic 
effects  of  these  covariates  with  concussion  on  post-concussive 
symptom  reporting. 

Effects  of  concussion  on  neurocognitive  functioning 

The  effects  of  concussion  on  neurocognitive  functioning  were 
analyzed  with  the  inclusion  of  covariates  including  previous  de¬ 
ployment,  CES,  PCL-M,  and  PHQ-8.  If  significance  was  marginal, 
analysis  was  conducted  for  PCL-M  and  PHQ-8  covariates  only  or 
with  no  covariates  and  reported  accordingly  to  determine  the  rel¬ 
ative  factors  influencing  neurocognitive  functioning.  Therefore, 
reported  significance  includes  all  covariates,  unless  otherwise  in¬ 
dicated.  Summary  of  findings,  including  Means,  SD,  %Correct,  and 
TP  along  with  significance  for  the  relevant  covariates  are  reported 
in  Table  3.  The  most  relevant  metric  for  analysis,  TP  for  SRT,  PRT, 
SPD,  CDS,  and  STN,  and  %Correct  for  GNG  and  CDD  are  used  for 
analysis,  below. 

Single  concussion.  Having  ever  experienced  a  previous 
concussion  (Y/N)  was  associated  with  SRT-TP  (F  =  7. 1 1 ,  p  =  0.008, 
Ifv = 0.01 1)  and  PRT-TP  (F= 5.82,  p  =  0.016,  ;;2p  =  0.009)  but  only 
without  covariates  included).  GNG-%,  CDS-TP,  SPD-TP,  STN- 
TP,  and  CDD-TP  were  not  significant,  even  without  inclusion  of 
any  covariates.  In  summary,  having  a  previous  concussion  (Y/N) 
was  not  associated  with  neurocognitive  outcomes  when  covariates 
for  PTSD,  PHQ,  and  deployment  experience  were  taken  into 
account. 

Recency  of  concussion .  The  effects  of  a  concussion  within  6 
months  compared  with  previously  or  never  concussed  was  signif¬ 
icant  for  SRT-TP  (F  =  4.48,  /?  =  0.012,  rj2p  =  0.014)  and  PRT-TP 
(F  =  4.81,/?  =  0.008,  rj2 p  =  0.015),  but  only  when  no  covariates  were 
included.  CDS-TP,  GNG-%,  SPD-TP,  STN-TP,  and  CDD-%  were 
not  significant,  even  without  including  co variates.  In  summary,  the 
effects  of  recency  of  concussion  were  detected  by  SRT  and  PRT 
only  when  no  covariates  were  included,  and  not  at  all  by  other 
neurocognitive  measures,  possibly  because  of  the  low  numbers  of 
service  members  who  reported  a  concussion  within  the  previous  6 
months.  Indeed,  cell  sizes  ranging  between  11  and  16  produced 
power  below  0.30  for  these  analyses. 

Multiple  lifetime  concussions.  Multiple  concussions  (0, 1, 2, 
3,  or  more)  were  associated  with  SRT-TP  (F  =  2.94,  /?  =  0.03, 
r\2 p  =  0.021,  with  only  PCL-M  and  PHQ-8  as  covariates),  margin¬ 
ally  for  PRT-TP  (F  =  2.57,  p  =  0.053,  rj2p  =  0.012,  without  covari¬ 
ates),  GNG-%  (F  =  3.15,/?  =  0.025,  rj2p  =  0.015,  without  covariates), 
and  CDS-TP  (F  =  2.68,  p  =  0.046,  rj2p  =  0.019,  without  covariates). 
CDS-%  SPD-TP,  SPD-TP,  and  CDD-%  were  not  significant,  even 
without  covariates.  Because  of  concerns  that  those  receiving  only 
Rapid  or  Brief  batteries  may  not  have  been  equivalent  to  those 
receiving  Standard,  similar  analyses  were  conducted  with  Brief 
subjects  only,  using  the  same  covariates  except  substituting  the 
PCPTSD  taken  by  Brief  subjects  instead  of  the  PCL-M. 

GLM  ANCOVA  with  covariates  entered  together  showed  that 
results  were  similar  for  this  subsample:  Number  of  Concussions, 
but  not  Recency  of  Concussion,  was  associated  with  SRT-TP,  PRT- 
TP,  GNG-%,  and  CDS-TP  without  covariates.  In  summary,  several 


of  the  neurocognitive  tests  requiring  simpler  processing  responses 
were  positive  for  multiple  concussion  without  covariates  included. 
In  all  cases,  results  were  in  the  expected  direction;  however,  when 
all  covariates  were  included,  these  trends  were  eliminated. 

3+  Lifetime  concussions.  Because  there  was  no  significant 
difference  between  none  and  one  concussions,  and  most  often  not 
even  with  two  concussions,  and  the  largest  difference  across  out¬ 
comes  was  from  having  three  or  more  lifetime  concussions,  addi¬ 
tional  analysis  was  conducted  on  three  or  more  concussions  (3  + ) 
compared  with  none  or  one  concussion  (0-1).  A  significant  dif¬ 
ference  was  detected,  incorporating  all  covariates,  for  SRT-TP 
(F  =  6.57,  /?  =  0.011,  //2p  =  0.017)  and  GNG-%  (F  =  4.43, /?  =  0.036, 
rj2 p  =  0.012)  incorporating  all  covariates,  PRT-TP  (F  =  3.76, 
p  =  0.05,  rj2 p  =  0.010,  with  PCL  and  PHQ8  covariates),  and  CDS-TP 
(F  =  6.16,/?  =  0.013,  rj2p  =  0.016)  with  the  inclusion  of  all  covariates. 
SPD-TP,  STN-TP  and  CDD-%  were  not  significant,  even  without 
the  inclusion  of  covariates.  No  interaction  of  recency  of  concussion 
by  3+  concussions  was  found.  In  summary,  those  who  reported 
three  or  more  lifetime  concussions  were  worse  on  neurocognitive 
tasks  requiring  simple  attention  and  simple  discrimination  skills, 
including  SRT-TP,  GNG-%,  and  CDS-TP.  The  effects  of  three  or 
more  concussions  were  not  detected  by  SPD-TP  or  the  memory 
tests  CDD-%  and  STN-%. 

Finally,  a  Binary  Logistic  Wald  Forward  Stepwise  Regression 
with  all  four  covariates  was  conducted  to  examine  the  effect  of 
three  or  more  compared  with  none  or  one  lifetime  concussions  on 
neurocognitive  functioning  further  than  one  SD  from  the  mean.  A 
z- score  of  <  1  SD  was  used  because  Receiver  Operating  Char¬ 
acteristic  Curve  Analysis  revealed  that  three  or  more  previous 
concussions  predicted  SRT-TP  <  1  SD  below  the  mean  with  a 
sensitivity  of  0.91  and  specificity  of  0.96,  and  an  overall  accuracy 
of  82%  (/?  <  0.03).  Analysis  was  limited  to  those  tests  shown  above 
to  be  sensitive  to  impairment:  SRT-TP,  PRT-TP,  and  %GNG  using 
a  cutoff  of  less  than  one  SD  from  the  mean,  and  including  cov¬ 
ariates  scores  on  PCL-M,  PHQ-8,  CES,  and  Previous  Deployment 
status. 

As  presented  in  Table  4,  three  or  more  lifetime  concussions 
predicted  neurocognitive  decrement  in  SRT-TP  (Wald  =  5.47, 
/?  =  0.019,  OR  =  2.17,  Cl:  1.16-5.24),  PRT-TP  (Wald  =  4.11, 
/?<  0.043,  OR  =  1.96,  Cl:  1.02-3.77),  and  GNG-%Correct 
(Wald  =  4.90,  /?<  0.027,  OR  =  2.82:  Cl:  1.13-7.06).  From  these 
results,  it  appears  that  simple  measures  of  neurocognitive  func¬ 
tioning  are  sensitive  to  the  effects  of  three  or  more  concussions, 
independent  of  emotional  and  deployment  co  variates. 

Discussion 

Given  the  spike  in  patients  presenting  with  concussions  to 
emergency  departments  over  the  past  5  years,  thought  to  be  because 
of  increased  public  awareness  of  the  problem,  understanding  per¬ 
sistent  symptoms  associated  with  concussion  is  of  growing  im¬ 
portance.52  Toward  that  end,  this  study  attempted  to  determine 
what  factors  are  related  to  PPCS,  especially  disentangling  de¬ 
ployment  factors  from  emotional  factors  from  concussion  type. 
PPCS  was  examined  in  terms  of  emotional  distress  (PTSD,  de¬ 
pression,  and  anger  symptoms),  post-concussive  symptoms  re¬ 
porting  (cognitive,  sensory,  somatic,  vestibular,  and  insomnia),  and 
neurocognitive  functioning  across  a  wide  range  of  domains  of 
speed  and  accuracy. 

This  research  was  distinct  from  earlier  studies  that  recruited 
patients  presenting  to  a  clinic  and  evaluated  a  narrower  range  of 
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Table  3.  Influence  of  Concussion  History  on  Cognitive  Tests 


Cognitive  tests  (sig.  with  all  covariates  included  or  none  as  indicated ) 


Tests 

N 

Mean  RT  (SD)  sig. 

%Correct  (SD)  sig. 

TP  (SD)  sig. 

SRT 

Any  past  concussion: 

No 

476 

332.1  (57.5) 

.990  (.03) 

193.4  (29.6) 

Yes 

167 

350.2  (83.6)*none 

.981  (.04)*none 

185.7  (38.1)*none 

Never 

494 

332.9  (59.0) 

.990  (.03) 

193.1  (29.9) 

>  6  mos.  ago 

136 

347.7  (82.7) 

.982  (.04) 

186.6  (37.8) 

<  6  mos.  ago 

14 

375.0  (17.5)*none 

.975  (.06)*none 

173.3  (42.7)*none 

0 

305 

336.0  (60.2) 

.999  (.04) 

190.7  (30.6) 

1 

35 

329.6  (60.7) 

.995  (.01) 

195.9  (20.3) 

2 

33 

349.5  (83.9) 

.980  (.05) 

184.7  (39.0) 

3  + 

42 

377.9  (107.3)*all 

.964  (.05)*all 

173.3  (47.0)* PCL/PHQ 

0-1 

340 

335.3  (60.2) 

.999  (.03) 

191.9  (30.5) 

vs  3  + 

42 

377.9  (107.3)***all 

.964  (.06)***all 

173.3  (47.0)** all 

PRT 

Any  past  concussion: 

No 

475 

629.0  (101.1) 

.985  (.02) 

101.5  (14.3) 

Yes 

168 

659.1  (152.6)*none 

.977  (.03)*none 

98.3  (16.5)*none 

Never 

493 

643.7  (106.6) 

.984  (.02) 

101.5  (14.2) 

>  6  mos.  ago 

138 

665.5  (156.4) 

.979  (.03) 

98.6  (16.0) 

<  6  mos.  ago 

15 

717.5  (263.3)**none 

.964  (.05)**none 

91.6  (22.4)**none 

0 

305 

643.7  (106.6) 

.984  (.02) 

99.8  (14.8) 

1 

36 

645.3  (119.5) 

.984  (.02) 

101.1  (16.0) 

2 

32 

651.27(93.3) 

.987  (.02) 

98.3  (12.7) 

3  + 

42 

709.6  (242.5)*none 

.966  (.03)***all 

94.1  (21.9)*none 

0-1 

341 

643.9  (107.9) 

.985  (.02) 

99.9  (14.9) 

vs  3  + 

42 

709.6  (242.5)*all 

.964  (.03)***all 

93.8  (2 1.7)* PCL/PHQ 

GNG 

Any  past  concussion: 

No 

462 

99.6  (14.8) 

.990  (.03) 

118.3  (19.1) 

Yes 

162 

97.7  (17.7)  n.s. 

.985  (.032)  n.s. 

116.8  (20.4)  n.s. 

Never 

479 

538.8  (101.7) 

.990  (.03) 

118.4  (19.0) 

>  6  mos.  ago 

131 

555.0  (135.4) 

.986  (.03) 

117.0  (20.3) 

<6  mos.  ago 

15 

568.5  (118.9)  n.s. 

.976  (.05)  n.s. 

111.3  (23.7)  n.s. 

0 

478 

538.8  (101.7) 

.990  (.03) 

118.4  (19.0) 

1 

51 

556.5  (126.4) 

.983  (.02) 

116.6  (19.7) 

2 

37 

552.3  (109.6) 

.984  (.03) 

115.9  (20.2) 

3  + 

58 

559.9  (110.1)  n.s. 

.979  (.04)* all 

116.5  (22.1)  n.s. 

0-1 

529 

540.5  (104.4) 

.989  (.03) 

118.2  (19.1) 

vs  3  + 

58 

559.0  (154.0)  n.s. 

.975  (.05)*all 

116.5  (22.1)  n.s. 

CDS 

Any  past  concussion: 

No 

302 

1346.4  (243.6) 

.966  (.04) 

46.9  (9.4) 

Yes 

115 

1394.9  (296.8)  n.s. 

.955  (.05)*none 

45.1  (10.5)  n.s. 

Never 

307 

1343.9  (243.3) 

.966  (.04) 

47.0  (9.4) 

>  6  mos.  ago 

96 

1388.5  (272.3) 

.958  (.04) 

45.2  (9.9) 

<  6  mos.  ago 

15 

1500.7  (422.4)*none 

.936  (.06)*none 

42.4  (9.7)  n.s. 

0 

305 

1344.0  (243.3) 

.966  (.04) 

47.0  (9.4) 

1 

36 

1332.6  (288.8) 

.960  (.04) 

47.4  (10.3) 

2 

32 

1435.7  (298.0) 

.952  (.05) 

44.1  (11.1) 

3  + 

43 

1439.3  (299.3)*none 

.942  (.05)  n.s. 

43.2  (9.7)*none 

0-1 

341 

1342.7  (248.0) 

.966  (.04) 

47.1  (9.5) 

vs  3  + 

43 

1439.4  (255.6)*none 

.952  (.05)  .n.s. 

43.2  (9. 7)* all 

SRT  Simple  Reaction  Time;  RT,  Reaction  Time;  SD,  standard  deviation;  TP,  throughput;  PCL,  Post-Traumatic  Symptom  Disorder  Check  List;  PHQ, 
Primary  Care  Health  Questionnaire;  PRT,  Procedural  Reaction  Time;  GNG,  Go-No-Go;  CDS,  Code  Substitution  Simultaneous. 

TP  is  a  measure  of  cognitive  efficiency,  based  on  both  speed  and  accuracy.  Higher  values  indicate  better  cognitive  efficiency. 

*p<0.05,  **/?<0.01,  ***p<  0.005  using  Previous  Deployment,  Combat  Exposure  Survey  (CES),  PCL,  PHQ  as  covariates. 

Number  of  concussions  (0, 1, 2, 3,  or  more)  and  Recency  of  Concussion  (less  than  or  greater  than  6  months  since  the  most  recent  concussion)  reporting  Mean 
Correct  RT,  SD,  Percent  Correct,  TP.  General  Linear  Model  (ANCOVA),  or  Generalized  Linear  Modeling  (for  unequal  variances)  results  for  significance  level 
with  covariates  are  reported.  Covariates  are  referred  to  as  ALL  (PCL/PHQ/CES/Deployment),  PCL/PHQ  (only),  and  none.  Memory  and  spatial  tests  (Spatial 
Rotation  Discrimination/  Sternberg  Memory  Test/  Code  Substitution-Delayed  Memory)  were  not  significant,  so  are  not  included  in  the  table. 
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Table  4.  Effect  of  Three  or  More  Lifetime  Concussions  on  Neurocognitive  Functioning 


Variables  in  the  equations 


SRT-TP  B 

S.E. 

Wald 

df 

Sig. 

Exp(B) 

Step  2b 

Conc_Count_01_v_3plus(l)  -.902 

.385 

5.477 

1 

.019 

2.174 

PCLOSCORE  -.027 

.010 

7.053 

1 

.008 

1.027 

Constant  2.236 

.414 

29.136 

1 

.000 

9.355 

a.  Variable(s)  entered  on  step  1:  PCLOSCORE. 

b.  Variable(s)  entered  on  step  2:  Conc_Count_01_v_3plus. 

PRT-TP  B 

S.E. 

Wald 

df 

Sig. 

Exp(B) 

Step  la 

Conc_Count_01_v_3plus(l)  .675 

.333 

4.113 

1 

.043 

1.964 

Constant  .048 

.166 

.084 

1 

.772 

1.049 

a.  Variable(s)  entered  on  step  1:  Conc_Count_01_v_3plus. 

GNG-%Correct  B 

S.E. 

Wald 

df 

Sig. 

Exp(B) 

Step  la 

Conc_Count_01_v_3plus(l)  1.037 

.468 

4.901 

1 

mi 

2.819 

Constant  -2.156 

.234 

84.805 

1 

.000 

.116 

a.  Variable(s)  entered  on  step  1:  Conc_Count_01_v_3plus. 

Binary  Logistic  Wald  Forward  Stepwise  Regression  examining  the  effect  of  three  or  more  lifetime  concussions  on  neurocognitive  functioning:  Simple 
Reaction  Time  Throughput  (SRT-TP),  Procedural  Reaction  Time  Throughput  (PRT-TP),  and  Go-No-Go  %Correct  (%GNG)  using  a  cutoff  of  less  than 
one  standard  deviation  from  the  mean,  and  including  covariates  scores  on  Post-Traumatic  Symptom  Disorder  Check  List-Military;  Primary  Care  Health 
Questionnaire;  Combat  Exposure  Survey,  and  Previous  Deployment  status.  Only  final  model  step  is  presented. 


outcomes  in  that  it  assessed  a  cohort  of  active  duty  US  military 
service  members  who  were  fit  for  duty  and  participating  in  daily 
work  activities,  with  assessment  of  previous  deployment,  combat 
exposure,  emotional  distress,  post-concussive  symptoms,  and 
neurocognitive  functioning  to  examine  persistent  effects  of  con¬ 
cussion.  Further,  by  including  previous  deployment,  combat  ex¬ 
posure,  PTSD,  and  depressive  symptoms  as  covariates  to  determine 
the  unique  contribution  of  concussion,  this  approach  was  able  to 
explore  the  theory  that  PPCS  is  because  of  emotional  issues  alone. 

Results  showed  that  recent  deployment,  higher  CES,  PCL-M, 
and  PHQ8  each  had  an  effect  on  concussion  number  and  recency, 
emotional  distress,  and  post-concussive  symptom  reporting,  as  well 
as  for  Attention-Discrimination  type  neurocognitive  tasks,  but  not 
for  memory-dependent  tasks.  The  presence  of  a  previous  concus¬ 
sion  as  well  as  recency  of  concussion  had  an  effect  on  emotional 
distress.  The  presence  of  a  previous  concussion  was  not  related  to 
post-concussive  symptom  reporting,  with  the  exception  of  insom¬ 
nia,  nor  to  neurocognitive  functioning.  Recency  and  number  of 
concussions,  however,  were  associated  with  post-concussive 
symptom  reporting,  and  number  of  concussions  was  related  to 
Attention-Discrimination  type  neurocognitive  functioning. 

Having  had  three  or  more  lifetime  concussions  was  associated 
with  the  worst  scores  for  depression,  PTSD,  anger,  post-concussive 
sensory,  somatic,  and  vestibular  symptoms,  and  was  predictive  of 
the  lowest  performers  on  simple  cognitive  tasks.  Although  sub¬ 
jective  cognitive  complaints  and  insomnia  symptoms  are  associ¬ 
ated  with  three  or  more  lifetime  concussions  with  a  moderate  effect 
size,  these  associations  were  not  sustained  when  deployment,  CES, 
PTSD  Sx,  and  Depressive  Sx  were  considered  as  covariates.  This 
suggests  that  persons  with  multiple  concussions  may  complain  of 
cognitive  and  insomnia  symptoms,  but  at  least  in  this  sample,  they 
appear  to  be  primarily  from  emotional  factors. 


Our  findings  support  earlier  reports  that  deployment  was  asso¬ 
ciated  with  worsening  neurocognitive32,40  and  emotional53  func¬ 
tioning.  We  also  found  support  for  previous  research  that 
depression  and  PTSD  are  strong  predictors  of  post-concussive 
symptoms,  including  neuropsychological  and  self-reported  post- 
concussive  symptoms.18,20  In  addition,  findings  in  this  study  sam¬ 
ple  for  the  most  part  supported  earlier  research  that  failed  to  find  the 
presence  of  a  previous  concussion  to  be  related  to  post-concussive 
symptom  reporting  or  a  decrease  in  neurocognitive  efficiency,  once 
the  contributions  of  deployment,  combat  exposure,  PTSD,  and 
depressive  symptoms  were  removed. 

The  presence  of  previous  concussion,  however,  independent  of 
previous  deployment  or  combat  exposure,  was  associated  with  worse 
levels  of  PTSD,  depression,  and  insomnia,  all  with  moderate  effect 
sizes.  Moreover,  the  presence  of  three  or  more  lifetime  concussions 
was  associated  with  a  decrease  in  cognitive  efficiency  and  an  increase 
in  post-concussive  symptoms,  independent  of  PTSD  and  depressive 
symptoms  or  likely  diagnosis.  Specifically,  recency  of  concussion 
was  associated  with  worse  emotional  and  post-concussive  symptoms, 
especially  for  those  who  had  three  lifetime  concussions. 

There  was  insufficient  power  to  determine  if  having  a  concus¬ 
sion  within  the  previous  6  months  was  a  further  contributor  to 
cognitive  functioning,  but  given  that  the  effects  of  multiple  con¬ 
cussion  were  consistently  worse  for  those  with  a  recent  deploy¬ 
ment,  and  the  interaction  of  recency  and  number  of  concussions 
in  other  domains,  the  impact  of  recency  of  concussion  on  post- 
concussive  symptoms  is  worthy  of  continued  examination.  Indeed, 
a  recent  study  of  active  duty  soldiers  showed  that  recent  concus¬ 
sion,  independent  of  injury  mechanism  or  combat  exposure,  is 
predictive  of  persistent  post-concussive  complaints.54 

Thus,  although  these  findings  are  consistent  with  research  that 
failed  to  find  independent  evidence  for  the  presence  or  absence  of 
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previous  concussion,  the  evidence  of  recent  and  multiple  concus¬ 
sion  across  a  range  of  emotional  distress,  post-concussive  symp¬ 
tom,  and  neurocognitive  functioning  is  in  contrast  with  other 
studies  that  failed  to  find  a  unique  contribution  for  concussion 
across  a  wide  range  of  persistent  post-concussive  symptoms.18,55 
Possible  reasons  for  this  may  include  the  failure  of  other  studies  to 
include  both  objective  and  subjective  assessments  of  functioning, 
assessment  of  a  wide  range  of  PPCS  including  neurocognitive,  self- 
report  emotional  and  somatic  symptoms,  and,  most  importantly,  the 
distinction  between  single  and  multiple  concussion,  and  its  re¬ 
cency.  Further,  the  assessment  of  deployment  and  combat  exposure 
has  rarely  been  used  to  tease  out  the  specific  effects  of  combat 
deployment  from  other  factors  influencing  PPCS  outcomes  in  ac¬ 
tive  duty  personnel.  This  may  have  implications  for  civilians  as 
well  in  that  severity  of  trauma  exposure  may  need  to  be  included  in 
any  assessment  of  PPCS. 

Although  we  included  previous  deployment  and  CES  as  covari¬ 
ates,  we  wanted  to  insure  that  the  results  based  on  the  entire  sample 
were  also  seen  in  the  recently  deployed  troops  only  (deployed  within 
the  previous  6  months).  Analysis  of  the  effects  of  three  or  more 
lifetime  concussions  in  this  subsample  showed  a  similar  pattern  to 
what  is  reported  with  the  entire  sample  on  emotional  and  post-con¬ 
cussive  symptoms  as  well  as  neurocognitive  functioning. 

Also  of  note,  recently  deployed  service  members  in  our  sample 
had  slower  reaction  times  across  several  neurocognitive  tests 
compared  with  never  or  previously  deployed  service  members. 
Yasterling  and  co workers, 32  however,  found  post-deployed  service 
members  responded  faster  in  several  neuropsychological  tests  than 
they  did  pre-deployment,  which  they  attributed  to  a  hyperaroused 
nervous  system.  Perhaps  this  is  because  of  differences  in  testing 
methods;  we  used  an  NCAT  capable  of  assessing  each  response  for 
speed  and  accuracy,  whereas  Vasterling  used  paper  and  pencil 
measures  to  determine  overall  speed.  Moreover,  these  authors  as¬ 
sessed  service  members  sooner  after  deployment,  when  their  ner¬ 
vous  systems  may  have  been  in  a  different  state  than  several  months 
later.  Certainly,  more  longitudinal  and  cross-sectional  research  on 
post-deployed  service  members  is  needed  to  shed  light  on  neuro¬ 
cognitive  performance  after  stressful  deployments. 

The  fact  that  this  neurobehavioral  battery  was  generally  sensi¬ 
tive  to  factors  (despite  that  covariates  often  accounted  for  most  of 
the  variance)  shows  that  such  neurobehavioral  batteries  hold 
promise  as  a  useful  tool  for  detecting  alterations  in  cognitive 
functioning  from  a  variety  of  reasons,  whether  emotional,  physical, 
or  because  of  fatigue.  It  is  noteworthy  that  those  tests  relying  more 
on  attention  and  simple  discrimination  (SRT,  PRT,  GNG,  and 
CDS)  were  more  sensitive  to  the  effects  of  recency  and  number  of 
lifetime  concussions  than  were  SPD  or  the  memory-dependent  tests 
(CDD,  STN).  This  may  be  because  of  the  increased  time  to  answer 
these  latter  tasks  along  with  the  ability  to  rally  compensatory  effort 
and  strategies.  By  contrast,  the  simpler  tasks  needed  to  be  answered 
in  a  faster  period  and  relied  on  more  primitive  processing  skills, 
thus  not  permitting  for  increased  effort  or  compensatory  mecha¬ 
nisms  to  be  used. 

Based  on  these  results,  this  study  shows  that  the  tests  contained 
within  this  neurobehavioral  battery,  especially  those  relying  on 
attention  and  simple  discrimination,  may  be  worthwhile  adminis¬ 
tering  to  military  members  to  determine  how  they  are  functioning 
so  that  early  detection  of  suspected  problems  can  aid  in  getting 
them  the  care  they  need.  Although  covariate  analysis  was  con¬ 
ducted  to  demonstrate  the  unique  contribution  of  concussion, 
PTSD,  depression,  and  other  factors  influencing  PPCS,  it  is  im¬ 
portant  to  appreciate  that  the  combination  of  concussive  and  psy¬ 


chological  factors  are  frequently  co-occurring,  likely  enhancing 
symptoms,  and  presenting  unique  challenges  to  diagnosis  and 
treatment. 

All  postconcussive  subjective  symptoms,  including  insomnia, 
seem  to  be  shared  by  multiple  concussion,  PTSD  symptoms,  de¬ 
pressive  symptoms,  and  combat  experience.  Yet  multiple  concus¬ 
sion  by  itself  appears  to  contribute  less  to  cognitive  complaints  and 
insomnia  than  it  does  to  sensory,  somatic,  and  vestibular  com¬ 
plaints.  This  is  a  unique  finding  in  the  literature,  possibly  because 
previous  studies  have  inadequately  controlled  for  covariates  that 
influence  these  subjective  symptoms.  If  this  finding  is  replicated  in 
other  samples,  then  these  complaints  may  need  to  be  more  heavily 
weighted  to  better  understand  the  unique  contribution  of  concus¬ 
sions  to  persistent  symptom  reporting. 

From  the  perspective  of  treating  persistent  postconcussive 
symptoms,  however,  distinguishing  between  the  independent  ef¬ 
fects  of  concussion  from  emotional  factors  may  not  always  be 
desired.  Given  that  concussion  is  associated  with  increased  odds  of 
PTSD  (OR  =  2.5),  depression  (OR  =  2.2),  and  anger  (OR  =  2.4), 
independent  of  combat  experience  (Table  1),  it  is  important  to 
appreciate  emotional  factors  as  frequently  comorbid  with  multiple 
concussion.  Thus,  clinicians’  consideration  of  the  effects  of  con¬ 
cussion  on  PPCS  for  determination  of  illness  severity,  and  possibly 
even  for  treatment  planning,  need  not  be  as  analytically  separated 
as  is  necessary  for  research. 

Research  to  improve  understanding  of  the  unique  contributions 
of  various  components  of  the  wounds  of  war,  as  well  as  ways  to 
better  appreciate  their  interaction  resulting  in  the  gestalt  experi¬ 
enced  by  the  veteran,  is  necessary.  Therefore,  although  it  is  im¬ 
portant  to  determine  the  unique  contributions  of  these  various 
factors,  it  is  also  important  to  appreciate  that  separating  them  may 
lead  to  an  underestimation  of  their  combined  effects. 

Based  on  these  results,  it  seems  premature  to  recommend  ces¬ 
sation  of  assessments  for  concussion  post-deployment.  Further, 
because  there  is  so  much  comorbidity  in  this  population,  further 
research  identifying  the  independent  and  combined  causes  of  per¬ 
sistent  symptoms  should  prove  useful  in  both  evaluating  the 
functioning  of  the  individual,  as  well  as  supporting  the  develop¬ 
ment  of  interventions  that  are  tailored  to  the  needs  of  those  with 
specific  symptom  clusters.56  Future  research  that  follows  a  cohort 
prospectively  to  better  understand  the  time-course  and  causality  of 
symptoms  associated  with  concussion,  as  well  as  identifying  if  the 
context  of  the  concussion  (such  as  combat,  assault,  or  accident), 
will  also  be  of  importance  in  the  understanding  and  treatment  of 
concussion  and  associated  symptoms. 
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Abstract 

A  durable,  portable,  and  field-hardened  computerized  neurocognitive  test  (CNT)  called  the 
Defense  Automated  Neurobehavioral  Assessment  (DANA)  tool  was  recently  developed  to  pro¬ 
vide  a  practical  means  to  conduct  neurological  and  psychological  assessment  in  situ.  The  psycho¬ 
metric  properties  of  the  DANA  have  been  previously  described.  This  present  work  discusses  the 
test-retest  reliability  of  the  DANA  Rapid  test  battery,  as  administered  to  a  homogeneous  popula¬ 
tion  of  U.S.  Air  Force  Academy  football  team  players  (N  =  1 62)  across  the  duration  of  the  season. 
The  intraclass  correlation  coefficient  (ICC)  metric  of  the  DANA  is  compared  with  that  from  two 
different  CNTs  recently  reported  in  Cole  et  al.,  and  the  implications  of  using  the  metric  to  inter¬ 
pret  comparative  test  reliability  among  different  CNTs  are  discussed. 

Keywords 

reliability,  cognitive  assessment,  screening,  traumatic  brain  injury,  concussion,  sports  concussion, 
CNT 

Introduction 

Although  the  efficiency  and  utility  of  a  neurocognitive  assessment  tool  is  determined  by  its 
internal  consistency,  and  the  measure  of  test— retest  reliability  is  typically  described  as  the  foun¬ 
dation  on  which  a  test’s  validity  is  established  (Cole  et  al.,  2013),  a  consistent  methodology  for 
quantifying  reliability  is  not  clearly  demonstrated  in  the  current  neuropsychology  literature. 
Disconcertingly,  recent  works  have  found  that  the  reliability  coefficients  of  computerized  neu¬ 
rocognitive  tests  (CNTs)  tend  to  fall  below  the  bar  of  “clinical  acceptability”  (Broglio,  Ferrara, 
Macciocchi,  Baumgartner,  &  Elliot,  2007;  Cole  et  al.,  2013;  Resch  et  al.,  2013;  Segalowitz  et 
al.,  2007).  Differences  in  the  characteristics  of  test  batteries,  differences  in  the  design  of  test— 
retest  studies,  and  insufficiently  explained  and  non- standardized  methods  of  analysis  have  all 
served  to  confound  the  matter  of  clearly  defining  a  quantifiable  measure  of  reliability  (Christie, 
Kamen,  Boucher,  Inglis,  &  Gabriel,  2010;  Weir,  2005). 
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The  definition  of  the  reliability  coefficient  is  both  contextual  and  application-specific  (e.g., 
Baumgartner,  1969;  Feldt  &  McKee,  1958;  Streiner  &  Norman,  1995),  and  as  a  result,  any 
given  reliability  coefficient  is  not  a  universal  measure  of  reliability.  For  example,  there  is  still  a 
pervasive  reporting  of  Pearson’s  r  to  assess  reliability  even  though  its  use  is  actively  discour¬ 
aged  due  to  the  model’s  inherent  disregard  for  systematic  error  (Baumgartner,  2000;  Bedard, 
Martin,  Krueger,  &  Brazil,  2000;  Kroll,  1962;  Ludbrook,  2002;  Safrit,  1976)  and  is  therefore  an 
inappropriate  metric  for  certain  applications. 

In  neuropsychology,  the  intraclass  correlation  coefficient  (ICC;  as  formalized  by  Shrout  & 
Fleiss,  1979,  and  then  updated  by  McGraw  &  Wong,  1996)  is  often  reported  as  the  stand-alone 
metric  of  test— retest  reliability.  By  definition,  the  ICC  calculation  entails  six  different  possible 
configuration  parameters  by  which  the  coefficient  is  determined,  and  each  model’s  estimate  is 
unique.  A  side-by-side  comparison  of  the  methods  of  recent  works  (such  as  Broglio  et  al.,  2007; 
Cole  et  al.,  2013;  Resch  et  al.,  2013)  reveals  that  the  ICC  approach  is  often  not  applied  in  a  stan¬ 
dardized  way,  possibly  because  the  ICC  model  itself  is  not  well-understood.  Implicit  disagree¬ 
ment  between  research  groups  pertaining  to  which  ICC  model  most  accurately  describes  the 
test— retest  design  may  stem  from  confusion  around  the  applicability  of  the  ICC  model  as  devel¬ 
oped  for  inter-rater  reliability  rather  than  for  test— retest  reliability  (Weir,  2005). 

These  two  types  of  reliability  describe  distinctively  different  situations:  Inter-rater  reliability 
tests  the  hypothesis  that  a  heterogeneous  group  of  judges  (the  raters)  similarly  rate  the  same  set 
of  subjects  (the  ratees)  across  multiple  testing  sessions.  Test— retest  reliability  tests  the  hypothesis 
that  the  subjects  themselves  (the  ratees)  perform  the  same  way  across  the  sessions  and  assumes 
the  raters  (the  computers)  are  the  same.  As  a  result,  the  ICC  was  designed  to  be  relatively  insen¬ 
sitive  to  within-subject,  session-to-session  variability,  and  is  thus  only  informative  of  test— retest 
reliability  by  proxy.  Owing  to  this  detail,  other  fields  (e.g.,  exercise  and  sports  science,  sports 
medicine,  and  physical  therapy)  report  the  ICC  along  with  a  precision  metric  provided  by  the 
standard  error  of  the  measurement  (SEM)  that  offers  an  absolute  bound  on  the  measurement  of 
interest  (Denegar  &  Ball,  1993;  Learmonth,  Dlugonski,  Pilutti,  Sandroff,  &  Motl,  2013).  It  is 
crucial  to  note  that  for  the  purpose  of  comparing  the  reliability  of  different  CNTs,  it  is  particu¬ 
larly  important  to  report  the  reliability  coefficient  accompanied  by  a  precision  metric. 

The  SEM  carries  the  same  units  as  the  measurement  of  interest  (e.g.,  throughput  or  reaction 
time)  and  it  is  informative  of  within-subject  reliability.  It  can  be  obtained  from  the  reliability  coef¬ 
ficient  (this  method  provides  the  coefficient’s  precision),  or  independently  of  the  reliability  coeffi¬ 
cient,  from  the  square  root  of  the  mean  square  error.  In  either  case,  the  minimum  difference  (MD) 
is  directly  constructed  from  the  SEM  and  describes  the  minimum  amount  of  change  in  results 
required  to  be  considered  a  real  effect  and  not  an  artifact  of  associated  error  (Weir,  2005). 

This  work  describes  consistent  test  and  retest  measurements  of  the  Defense  Automated 
Neurobehavioral  Assessment  (DANA)  test  as  administered  to  a  homogeneous  population  of 
U.S.  Air  Force  Academy  football  team  players  ( N  =  162)  through  time  points  across  the  season. 
The  ICC  metric  of  the  DANA  is  compared  with  that  from  two  different  CNTs  recently  reported 
in  Cole  et  al.  (2013),  and  the  implications  of  using  the  metric  to  interpret  comparative  reliability 
among  different  CNTs  are  discussed. 

Method 

The  DANA  Rapid  Tests  and  Administration 

The  psychometric  properties  of  the  DANA  test  batteries  have  been  previously  described  and 
evaluated  (Lathan,  Spira,  Bleiberg,  Vice,  &  Tsao,  2013).  Data  for  this  study  were  collected 
under  the  U.S.  Air  Force  Academy  performance  improvement  protocol.  The  DANA  Rapid  test 


Downloaded  from  apm.sagepub.com  by  guest  on  March  30,  2015 


Russo  and  Lathan 


3 


Table  1 . 

Summary  Statistics  for  Each  Testing  Session  Date  (Denoted 

as  T|  >2). 

Battery 

Subtest 

<TP> 

SD  <TP> 

n 

R 

MD 

Ti 

T2 

7 1 

T2 

LB 

UB 

DANA 

PRT 

102.9 

106.8 

13.02 

12.57 

89 

0.75 

18.04 

0.60 

0.84 

RT 

192.4 

198.3 

26.77 

23.54 

87 

0.81 

32.34 

0.69 

0.87 

ANAM4 

PRT 

97.62 

103.56 

13.10 

10.31 

50 

0.51 

25.42 

0.28 

0.69 

RT 

86.88 

91.38 

1  1.46 

12.76 

50 

0.60 

22.37 

0.39 

0.75 

ImPACT 

RT 

0.60 

0.61 

0.08 

0.09 

44 

0.53 

0.17 

0.28 

0.71 

Note.  Reported  mean  throughput  (<TP>)  is  the  mean  throughput  of  n  total  subjects  (matched  for  test  and  retest), 
and  SD  <TP>  is  the  associated  standard  deviation  of  the  mean  throughput  of  n  total  subjects.  For  the  DANA  data,  R 
was  calculated  by  the  way  of  Equation  2,  presented  with  LB  and  UB  (constructed  in  the  95%  confidence  interval). 

With  respect  to  the  ANAM4  and  ImPACT  data  listed  here  from  Cole  et  al.  (2013),  the  ICC  model  used  to  calculate  R 
which  was  not  reported  in  that  work.  Note  that  the  reported  metric  for  ImPACT  is  not  <TP>,  but  rather  ImPACT’s 
composite  score.  The  MD  is  calculated  from  Equations  3  and  4,  using  the  greater  standard  deviation  between  the  two 
time  points  per  subtest,  per  test  battery.  TP  =  throughput;  LB  =  lower  bound;  UB  =  upper  bound;  MD  =  minimum 
difference;  DANA  =  Defense  Automated  Neurobehavioral  Assessment;  ANAM  =  Automated  Neurobehavioral 
Assessment;  ImPACT  =  Immediate  Post-Concussion  Assessment  and  Cognitive  Testing;  RT  =  Reaction  Time; 

PRT  =  Procedural  Reaction  Time. 


battery  consists  of  three  cognitive  tests  given  in  succession,  each  of  which  measures  reaction 
time  (see  Table  I  in  Lathan  et  al.,  2013).  On  a  given  testing  date,  U.S.  Air  Force  cadets  partici¬ 
pating  on  the  Air  Force  Academy  football  team  were  administered  the  DANA  Rapid  along  with 
a  demographic  survey  that  were  both  loaded  onto  a  collection  of  identical  mobile  devices 
(Trimble  Nomads,  model  900S).  The  test  administration  time  totaled  about  5  min. 

Data  were  collected  at  the  beginning  of  the  season  on  August  22  to  24,  2012,  in  the  middle 
of  the  season  on  November  6  to  7,  2012,  and  at  the  end  of  the  season  on  April  30  to  May  1, 
2013.  If  a  subject  took  more  than  one  administration  of  a  test  in  any  given  testing  session,  then 
only  the  first  administration  was  included  in  the  following  analysis.  In  addition,  a  subject  must 
have  correctly  responded  to  more  than  66%  of  the  test  stimuli.  Test— retest  reliability  was  calcu¬ 
lated  from  the  scores  of  the  first  test  (or  only  test)  administered  per  testing  date,  tabulated  for 
the  same  subjects  across  the  season.  The  test— retest  reliability  (reported  ICC)  of  the  DANA  is 
compared  with  that  from  two  of  the  CNTs  reported  in  Cole  et  al.  (2013),  for  comparable  neuro- 
cognitive  subtests.  The  retest  duration  specified  in  Cole  et  al.  was  21  to  42  days  post-baseline, 
and  thus  for  better  comparison,  the  shortest  period  retest  of  the  DANA  that  was  administered  is 
reported,  approximately  77  days  post-baseline  ( August-No vember  administrations). 

For  each  of  n  subjects,  a  subject’s  mean  throughput  (<TP>,  with  units  of  min-1)  was  calcu¬ 
lated  from  correctly  answered  mean  response  time  data  (<RTcorrect>,  with  units  of 
milliseconds), 


total  cotTect 

<TP>= - ^ - X  60, 000.  (1) 

<RTcorrect-> 

The  factor  of  60,000  converts  milliseconds  to  minutes.  To  assess  consistency  in  <TP>  in 
the  DANA  data  across  time  points,  a  multi-way  repeated-measures  ANOVA  was  performed  for 
all  of  the  subjects  across  testing  sessions,  per  subtest. 
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The  ICC 

In  essence,  the  ICC  is  a  relative  (unitless)  measure  of  test  error  with  regard  to  between-subject 
variability.  The  interpretation  of  the  ICC  is  that  it  represents  the  score  variance  that  is  attributa¬ 
ble  to  the  variability  between  subjects  and  the  remainder  of  the  variance  is  attributable  to  error. 
Because  systematic  differences  can  explain  variability  in  the  data,  the  choice  of  ICC  model 
parameters  is  dependent  on  the  study  design  and  the  characteristics  of  the  test  subjects.  Test— 
retest  designs  lend  themselves  to  two-way  model  analyses,  and  for  situations  in  which  the  test¬ 
ing  devices  are  not  themselves  variable  (identical  and  electronically  stable),  use  of  the  average 
measures  parameter  is  appropriate  as  it  will  not  over- compensate  for  error  and  thus  artificially 
deflate  the  resulting  coefficient  (Christie  et  al.,  2010;  Weir,  2005).  The  two-way,  average  mea¬ 
sures  model  is  given  as, 


ICC{2,k}  = 


MSs  —  MS^ 

MS  k(MST-M^ 

^  n 


(2) 


where  MS s,e,t  denote  the  mean  squares  (derived  from  the  sum  of  squares  resulting  from  a 
repeated-measures  ANOVA  calculation)  of  subjects  ( S ),  systematic  error  ( E ),  and  trials  (7);  k  is 
the  number  of  test  administrations  and  n  is  the  number  of  subjects.  On  the  DANA  data  pre¬ 
sented  in  this  work,  the  ICC  was  tabulated  using  a  MATLAB  script  (Salarian,  2008)  that  follows 
the  formalism  in  McGraw  and  Wong  (1996). 


SEM  and  MD 

The  SEM  is  an  absolute  index  of  reliability  and  provides  insight  into  the  session-to-session  noise 
in  a  given  set  of  data.  It  carries  the  same  units  as  the  measurement  of  interest  and  can  be  inter¬ 
preted  as  the  reliability  within  individual  subjects  (Shrout,  1998;  Weir,  2005).  The  SEM  can  be 
found  both  from  the  ICC,  or  estimated  independently  from  the  ICC  as  the  square  root  of  the 
mean  square  error  (MS^;  Eliasziw,  Young,  Woodbury,  &  Fryday-Field,  1994;  Hopkins,  2000; 
Stratford  &  Goldsmith,  1997;  Weir,  2005).  Following  the  formalism  in  Weir  (2005),  from  the 
ICC,  the  SEM  is  written  as, 


SEMicc  =  SDV 1  —  ICC, 


(3) 


where  SD  is  the  standard  deviation  of  subjects’  scores. 

Alternatively,  the  ICC-independent  form  of  the  SEM  is  calculated  using  the  MS^, 
related  to  the  sum  of  squares  error,  SS^,  found  from  the  ANOVA  calculation, 
MS^  =  SS^/((^z  —  1  ){k  —  1)),  where  n  is  the  number  of  subjects  and  k  is  the  number  of  test 
administrations.  The  ICC-independent  form  of  the  SEM  is  the  square  root  of  the  MS^ 
(SEM  =  ^fS e). 

In  either  case,  the  SEM  is  the  basis  of  the  MD  index  or  the  minimum  increment  of  observable 
change  that  warrants  consideration  as  a  real  change  in  score  and  likely  not  attributable  to  error: 

MD  =  SEM  XzX  \/2,  (4) 

where  the  \/2  is  an  artifact  of  the  standard  error  of  the  difference  of  two  score  results  from  test 
and  retest  administrations.  In  Equation  4,  z  is  the  distribution  score  used  to  construct  the  confi¬ 
dence  interval.  In  this  work,  the  MD  is  reported  as  an  absolute  index  for  reliability,  constructed 
in  the  95%  confidence  interval  for  which  z  =  1.96. 
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Results  and  Discussion 

The  results  from  all  tested  batteries  are  captured  in  Table  1.  The  reliability  coefficient  measured 
for  DANA,  for  matching  subjects  across  test  and  retest  sessions,  is  found  to  be  higher  than  those 
from  both  the  Automated  Neurobehavioral  Assessment  Metric  (ANAM),  and  the  Immediate 
Post-Concussion  Assessment  and  Cognitive  Testing  (ImPACT)  tools,  and  comparatively,  the 
DANA  exhibits  a  reliability  coefficient  within  tighter  upper  and  lower  bounds  than  both  ANAM 
and  ImPACT.  A  multi-way  repeated-measures  ANOVA  was  performed  for  all  of  the  subjects 
per  DANA  subtest,  grouped  by  testing  session,  and  the  results  (not  reported,  for  brevity)  showed 
that  the  marginal  means  of  <TP>  in  the  retest  sessions  are  not  significantly  different  from  those 
in  the  first  test  sessions  (p>  .05).  For  a  homogeneous,  non-clinical  population,  the  average 
throughput  per  subtest  is  not  expected  to  change  across  testing  sessions;  in  other  words,  the 
expectation  for  test  reliability  as  performed  on  healthy  and  unvaried  subjects  is  that  the  results 
of  the  retest  administration  should  show  no  statistically  significant  change  from  the  results  of 
the  first  test  administration.  The  result  of  the  ANOVA  is  interpreted  as  a  demonstration  of  con¬ 
sistency  across  test  and  retest  sessions. 

The  tabulated  MD  for  each  subtest  is  based  on  the  reported  ICC  coefficient  for  each  test  and 
battery,  and  the  greater  reported  standard  deviation  between  the  two  time  points.  In  this  context, 
the  MD  can  be  interpreted  as  the  minimum  amount  of  change  in  the  population’s  score  neces¬ 
sary  to  alert  a  statistically  significant  decrease  or  increase  in  test  performance.  The  MD  for 
DANA  per  subtest  is  approximately  17%  of  the  <TP>,  whereas  the  MD  of  both  ANAM  and 
ImPACT  are  approximately  24%  and  27%  of  the  <TP>,  suggesting  that,  for  the  timescale 
reported  here,  DANA  is  a  more  sensitive  measure  to  changes  in  test  performance.  In  other 
words,  a  smaller  proportional  MD  suggests  that  the  measurement  it  describes  is  more  sensitive 
to  change  and  less  confounded  by  error  than  a  measurement  with  a  proportionally  larger  MD. 

The  reliable  change  index  (RCI)  has  been  suggested  as  an  alternative  method  to  test— retest 
reliability,  and  several  methods  for  determining  the  RCI  have  been  published  previously  (e.g., 
Bruggemans,  de  Vijver,  &  Huysmans,  1997;  Chelune,  Naugle,  Luders,  Sedlack,  &  Awad,  1993; 
Jacobson  &  Truax,  1991;  Moritz,  Iverson,  &  Woodward,  2003;  Temkin,  Heaton,  Grant,  & 
Dikmen,  1999).  Each  of  these  methods  distinctly  relies  upon  the  calculation  of  a  reliability  coef¬ 
ficient,  arrived  at  by  way  of  the  ICC  or  by  Pearson’s  r,  or  both.  The  aforementioned  issues  that 
arise  from  a  non-standardized  approach  to  arriving  at  the  reliability  coefficient  serve  to  confound 
the  interpretation  of  a  RCI,  specifically  if  more  than  one  measure  of  reliability  is  calculated  for 
any  given  test.  For  example,  two  measures  of  reliability  are  presented  in  Cole  et  al.  (2013)  and 
the  standard  error  of  the  difference  (S&s,  a  precursor  to  the  RCI)  is  tabulated  using  whichever 
measure  is  greater.  When  distinct  disagreement  exists  between  each  measure  within  a  given  test, 
and/or  when  comparing  tests  that  were  measured  using  different  reliability  models,  this  approach 
is  misleading.  Under  such  circumstances  and  if  the  ICC  is  the  calculated  reliability  measure,  the 
ICC-dependent  form  of  the  SEM  can  provide  an  absolute  bound  on  the  reliability  coefficient,  or 
alternatively,  the  ICC-independent  form  of  the  SEM  provides  a  bound  on  the  measurement  error. 
In  either  case,  the  resulting  MD  is  found  to  be  a  consistent  index  for  reliable  change. 

Conclusion 

The  DANA  Rapid  test  battery  was  administered  to  cadets  of  the  U.S.  Air  Force  Academy  foot¬ 
ball  team  before  the  commencement  of  the  season  and  again  during  the  season.  The  test— retest 
reliability  of  the  DANA  tool  was  evaluated  and  it  was  found  that  the  test  is  consistent  between 
test  and  retest  sessions  administered  within  approximately  77  days.  Owing  to  the  non- 
standardized  practices  of  measuring  test— retest  reliability,  both  reporting  the  ICC  along  with  a 
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precision  metric  and  utilizing  alternative  methods  to  arrive  at  a  RCI  are  suggested,  such  as  with 
the  MD  approach,  which  is  described  in  the  “SEM  and  MD”  section. 
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Introduction 


Caffeine  is  commonly  used  to  alleviate  the  effects  of  sleep  deprivation.  A 
specific  population  that  must  perform  while  sleep  deprived  is  the  military.  Many 
military  operations  require  convoys  to  travel  across  flat  desert  terrain  in  order  to 
accomplish  various  tasks.  There  are  specific  regulations  as  to  the  distance 
between  vehicles  that  must  be  maintained,  and  speed  at  which  vehicles  can  be 
driven  within  these  convoys.  Driving  with  a  specific  distance  between,  a  slower 
speed  and  in  an  unaesthetic,  repetitive  landscape  can  become  boring  and 
monotonous.  If  a  fatigued  military  truck  driver  is  unable  to  stay  alert  on-duty  the 
safety  of  the  driver,  the  passengers  and  the  rest  of  the  convoy  is  at  risk.  This  study 
serves  to  characterize  the  effects  of  sleep  deprivation  with  dull  tasks  and  examine 
the  ability  of  caffeine  to  mitigate  these  effects.  The  Defense  Automated 
Neurobehavioral  Assessment  (DANA)  Rapid  battery  and  a  driver  simulator  were 
employed  to  measure  the  effectiveness  of  a  multiple-dose  caffeine  administration 
paradigm  to  maintain  vigilance  during  2  nights  of  sleep  deprivation.  This  poster 
focuses  primarily  on  performance  of  the  DANA  tasks. 


Methods 


Participants  and  Design 

Participation  in  this  study  was  voluntary  and  informed  consent  was 
received  from  each  subject.  The  study  was  proctored  as  a  double-blind  group 
design,  in  which  24  participants  were  randomly  assigned  into  6  groups  of  4.  There 
were  two  possible  conditions  to  which  the  groups  could  be  assigned:  placebo  or 
caffeine.  Each  group  was  randomly  assigned  to  receive  a  caffeine  or  placebo 
administration. 

Study  subjects  spent  a  total  of  4  days  and  3  nights  in  the  sleep  laboratory. 
Subjects  were  given  a  baseline  sleep  saturation  from  10pm  to  8am,  which  was 
followed  by  49  hours  of  sleep  deprivation/wakefulness.  Caffeine  was 
administered  in  2-hour  intervals— specifically  at  0100,  0300,  0500,  and  0700— on 
both  nights  of  sleep  deprivation.  Performance  results  from  the  caffeine  group 
were  compared  to  results  from  the  placebo  group. 

The  primary  task  subjects  participated  in  was  a  40-minute  driving  simulator 
(SIM)  designed  to  mimic  the  Australian  outback.  The  simulator  has  seats  mounted 
on  motion  actuators.  This  specific  environment  was  chosen  because  the  terrain  is 
flat,  repetitive  and  barren.  The  monotony  in  this  simulation  allows  a  more  accurate 
examination  of  caffeine’s  ability  to  diminish  the  effects  of  sleep  deprivation  when 
performing  a  boring  task. 

After  completing  the  driving  simulation,  each  group  performed  a  5-minute 
PVT  followed  by  the  DANA  Rapid.  The  DANA  Rapid  included  a  battery  of  three 
neurocognitive  assessments  to  note:  Simple  Reaction  Time  (SRT),  Procedural 
Reaction  Time  (PRT),  and  Go/No  Go  (GNG).  There  were  a  total  of  18 
administrations  per  subject  and  the  last  administration  from  Day  1  was  used  as 
the  baseline.  Placebo  and  caffeine  groups  were  evaluated  for  statistical 
significance  in  reaction  time  needed  to  make  correct  responses  and  the  number 
of  response  lapses  for  each  test.  A  lapse  for  the  SRT  was  >900ms;  for  PRT 
>2000ms  and  for  GNG  >1500ms.  For  the  SIM  task  the  number  of  crashes  during 
each  session  was  used  as  the  primary  variable  for  this  presentation. 

Measures 

Defense  Automated  Neurobehavioral  Assessment  -  Rapid  (DANA)  -Subjects 
completed  the  Rapid  battery  which  consists  of  assessments  of  Simple  Reaction 
Time  (SRT),  Procedural  Reaction  Time  (PRT),  and  a  Go-No-Go  (GNG)  task. 

Driving  simulator  -  Subjects  completed  a  40  minute  driving  simulation  that  was 
designed  to  reflect  the  environment  in  which  on-duty  military  personnel  are  tasked 
to  drive  in.  The  simulator  was  a  full  motion  platform  with  audio  ques. 


Figure  A-D  (to  be  read  left  to  right) 

A)  SRT  among  caffeine  and  placebo  group  are  comparable  until  subjects  are  sleep  deprived.  The  placebo  condition  jumps  from  0  lapses  in  the  8th  DANA 
administration  to  nearly  6  lapses  in  the  9th  The  caffeine  condition  holds  constant  between  the  8th  and  9th  administrations.  From  the  8th  test  administration 
through  the  18th  both  conditions  increase  their  number  of  lapses  but  the  placebo  group  has  more  lapses  than  the  caffeine  at  every  administration. 

B)  PRT  is  the  same  between  placebo  and  caffeine  conditions  for  the  first  two  days.  The  placebo  condition  has  0  lapses  at  the  8th  administration  and  3  lapses  by  the 
10th.  The  caffeine  condition  holds  0  lapses  until  the  14th  administration  where  <1  lapse  occurs.  At  every  administration  point  during  sleep  deprivation  the 
placebo  group  has  more  lapses  than  the  caffeine  group. 

C)  GNG  experienced  the  least  obvious  differences  between  conditions.  While  there  is  still  an  increase  in  lapses  seen  within  the  placebo  condition  compared  to  the 
caffeine  condition,  the  differences  are  smaller.  Between  administrations  8  and  14  there  is  the  greatest  difference  between  conditions  with  placebo  having  2 
lapses  in  GNG  at  administration  10  and  caffeine  having  0  lapses  at  the  same  gathering  point.  At  administration  16  placebo  and  caffeine  groups  are  most  similar 
during  the  sleep  deprivation  period,  seeing  1  lapse  each. 

D)  Figure  D  takes  measurements  from  two  points-in-time  to  see  if  the  number  of  SRT  lapses  gathered  from  the  DANA  Rapid  correspond  to  the  number  of  crashes 
seen  in  the  driving  simulation.  At  baseline,  both  conditions  are  performing  with  <1  lapse  or  crash.  At  day  3  the  placebo  condition  has  3  lapses  in  SRT 
performance  and  7  crashes  in  the  driving  simulation,  while  the  caffeine  condition  still  holds  <1  lapse  and  crash.  The  placebo  condition  had  a  6-fold  increase  in 
their  number  of  lapses  in  SRT  performance  and  a  7-fold  increase  in  their  number  of  crashes  on  day  3,  as  compared  to  baseline. 

Lapses  for  the  SRT  are  responses  >900ms;  for  PRT  >2000ms  and  for  GNG  >1500ms. 

*SRT,  PRT  and  GNG  defined  under  measures 
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Results 


Repeated  measures  ANOVA  (analysis  of  variance)  were  used  in  order  to 
determine  a  difference  in  performance.  Difference  in  performance  was  measured 
by  evaluating  mean  reaction  time  of  correct  responses  (DANA)  and  number  of 
crashes  between  the  placebo  and  caffeine  groups  (SIM).  Additionally,  logistic 
regressions  were  used  on  lapsed  responses  in  order  to  determine  the  odds  ratio 
when  given  placebo  compared  to  caffeine. 

For  the  DANA  subtests,  the  repeated  measure  ANOVA  identified  a 
statistically  significant  difference  in  mean  reaction  time  between  placebo  and 
caffeine  groups.  For  both  SRT  and  PRT,  subjects  in  the  caffeine  condition 
performed  faster  by  28  ms  (p=0.0433)and  68  ms  (p=0.0216),  respectively  (See 
Figs  A  &  B).  While  the  number  of  lapsed  responses  was  higher  in  the  placebo 
group  for  the  GNG,  this  was  not  significant  (Fig  C). 

The  logistic  regression  analysis  showed  significant  results  for  all  three 
subtests.  Subjects  were  4  times  more  likely  to  have  a  lapsed  response  for  SRT,  5 
times  more  likely  for  PRT,  and  3  times  more  likely  for  GNG.  Towards  the  end  of  the 
second  night  many  of  the  subjects  needed  to  be  woken  up  to  take  the  DANA. 

Currently,  only  data  from  sessions  1  and  9  were  available  for  analysis  from 
the  driving  simulation.  For  the  driving  simulation,  the  repeated  measures  ANOVA 
resulted  in  a  statistically  significant  difference  in  the  number  of  crashes  between 
placebo  and  caffeine  groups  (p=0.0154).  As  illustrated  in  Fig  D  the  placebo  group 
committed  3  more  crashes  (mean)  as  compared  to  the  caffeine  group.  The  logistic 
regression  analysis  was  also  significant  with  placebo  subjects  23  times  more 
likely  to  have  crashed  as  compared  to  the  caffeine  group. 


Summary/  Discussion 


Figures  A-C  compare  placebo  and  caffeine  groups  by  their  respective 
number  of  lapsed  responses.  Groups  were  analyzed  by  lapses  rather  than  speed 
because  lapses  are  an  accepted  indicator  of  when  a  task  cannot  be  performed 
correctly.  Speed  analysis  focuses  on  the  amount  of  time  it  takes  to  make  the 
correct  response,  but  does  not  evaluate  incorrect  responses,  which  leaves  a 
considerable  amount  of  data  unrepresented.  There  is  a  speed-accuracy  tradeoff 
in  human  behavior  where  people  will  reduce  speed  in  order  to  increase  their 
accuracy.  An  analysis  using  lapses  account  for  this  adjustment,  while  a  speed 
analysis  does  not. 

Even  though  the  effects  of  SRT  and  GNG  are  statistically  significant,  they 
may  not  be  operationally  significant  since  the  subjects’  performance  declined  by 
less  than  50ms  in  the  post  treatment  measures.  However,  this  may  be  due  to  the 
use  of  only  the  correct  responses  for  this  measure.  It  is  also  possible  that  this 
small  difference  is  due  to  the  nature  of  going  from  a  monotonous  driving  task  to  a 
different  task  that  required  an  outside  stimulus  (i.e.  being  handed  a  device)  which 
has  an  awakening  quality  to  it.  This  non-significance  could  also  be  due  to  small 
sample  size. 

An  increased  number  of  lapses  signals  decreased  attention  during  a  task. 
There  was  a  6  fold  increase  in  the  number  of  lapses  between  days  1  and  3  in  SRT 
performance  for  the  placebo  condition.  If  3  lapses  were  seen  during  the  SRT  task 
(which  runs  for  2minutes)  on  day  three,  we  could  expect  to  see  60  lapses  during 
the  40  minute  driving  simulation.  The  data  gathered  from  sessions  1  and  9 
suggests  an  increase  in  SRT  performance  lapses  could  indicate  an  increase  in 
poor  driving  performance  due  to  sleep  deprivation.  On  day  3  the  placebo  condition 
saw  a  7-fold  increase  in  the  number  of  crashes  during  the  driving  simulation,  as 
compared  to  their  baseline  performance  on  day  1.  While  lapses  in  the  DANA  do 
not  indicate  crashes,  they  do  indicate  decreased  performance.  If  data  from 
subsequent  sessions  supports  what  we  report  here  then  performance  on  the 
DANA  may  be  useful  as  an  indicator  of  driving  performance  while  sleep  deprived. 

It  is  important  to  note  that  some  of  the  placebo  subjects  were  experiencing 
micro  sleep  and  required  awakening  at  the  end  of  the  SIM  task  in  order  to 
complete  the  PVT  and  DANA.  This  arousal  could  have  altered  the  number  of 
lapses  that  would  have  otherwise  been  seen  and  may  have  had  an  impact  on 
DANA  performance.  Also,  there  is  a  certain  degree  of  intrapersonal  variation 
within  test  subjects.  Some  subjects  may  have  been  more  or  less  caffeine  sensitive 
than  others  as  well  as  their  individual  sensitivity  to  sleep  deprivation.  These 
differences  may  account  for  how  the  variation  in  performance  observed  here. 

In  conclusion,  even  after  two  nights  of  sleep  deprivation,  caffeine 
maintained  performance  on  the  DANA  tasks  and  simulator  driving  performance  as 
compared  to  placebo. 
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The  Defense  Automated  NeuroBehavioral  Assessment  (DANA)  is  a  clinical  assessment  tool  that  an  AnthroTronix-led  team  developed  for  the  Department  of  Defense 


Introduction 


Figure  1:  Screen  shots  for  DANA  cognitive  assessments 


Major  Depressive  Disorder  (MDD)  in  active  military  personnel  and 
veterans  has  been  a  major  source  of  morbidity  and  mortality.  MDD 
is  often  associated  with  cognitive  impairment1.  Little  is  understood 
about  the  cognitive  effects  of  electroconvulsive  therapy  (ECT) 
during  the  treatment  course  among  patients  with  MDD.  Our  study 
represents  a  novel,  automated  assessment  of  the  effect  on 
cognitive  functioning  of  patients  receiving  ECT  treatment  for  MDD. 


Objectives 


The  objectives  of  this  study  were  to  measure  cognitive  changes  in 
MDD  patients  during  their  course  of  ECT  treatment.  We  correlated 
cognitive  measures  obtained  by  administering  DANA  with  results  of 
Mini  Mental  Status  Examinations  (MMSE). 


Materials  and  Methods 


The  neurocognitive  and  psychological  assessment  battery  was  conducted  on  a 
32GB  Samsung  Galaxy  v8.9  electronic  tablet  with  Defense  Automated 
Neurobehavioral  Assessment  (DANA)  software.  DANA  is  a  new  neurocognitive 
assessment  tool  that  includes  a  library  of  standardized  cognitive  and 
psychological  assessments2-3. 

The  tablet  is  easy  to  use,  portable,  reliable,  and  provides  objective, 
immediate,  and  quantitative  data.  All  tests  included  in  the  DANA  assessment 
meet  the  requirements  of  the  American  Psychological  Association's  standard 
for  tests  and  measurements  and  all  tests  are  in  the  public  domain. 
Components  of  the  DANA  cognitive  assessment  battery  used  for  the  study 
were  administered  in  the  following  order:  simple  reaction  time  1  (SRT1), 
procedural  reaction  time  (PRT),  code  substitution  learning  simultaneous 
(CDS),  spatial  discrimination,  Sternberg  memory  search  (STN),  and  simple 
reaction  time  2  (SRT2)  (Figure  1).  Data  were  obtained  from  cognitive 
assessments  administered  twice  per  week  and  Mini-Mental  Status  Exam 
(MMSE)  assessments  administered  2-3  times  per  week  (Table  2). 

Inclusion  Criteria:  Patients  with  major  depression  undergoing  ECT  for  major 
depression  on  the  psychiatry  unit  at  the  Johns  Hopkins  Hospital  were  included 
in  the  current  study.  Patients  were  either  male  or  female,  ages  18  and  above. 
Exclusion  Criteria:  Demented  patients  incapable  of  completing  DANA 
cognitive  assessments  were  be  excluded  from  the  study,  as  were  patients 
with  active  comorbid  substance  abuse  disorders. 

This  study  was  approved  by  the  Johns  Hopkins  IRB  and  The  US  Army  Medical 
Research  and  Materiel  Command  (USAMRMC),  Office  of  Research  Protections 
(ORP),  Human  Research  Protection  Office  (HRPO)  approved  the  subject 
protocol  on  4  November  2013. 


Simple  Reaction  Time  (SRT) 

•  The  subject  taps  on  the  location  of 
the  yellow  symbol  as  quickly  as 
possible  each  time  it  appears. 

•  Assesses  pure  reaction  time. 


Procedural  Reaction  Time 

•  The  screen  displays  one  of  four  numbers  of 
3  seconds. 

•  Subject  presses  on  a  left  button  ("2"  or 
"3")  or  right  button  ("3"  or  "4")  depending 
on  the  number  presented. 

•  Choice  RT  measure  of  accuracy,  RT,  & 
impulsivity  which  targets  simple  executive 
functioning  with  easy  decision-making 
capabilities 


Code  Substitution  Learning 

•  Subjects  refer  to  a  code  set  of  9  symbol-digit 
pairs  that  are  shown  across  the  upper  portion 
of  the  screen. 

•  A  sequence  of  single  symbol-digit  pairs  is 
shown  below  the  key,  and  the  subject 
indicates  whether  or  not  the  single  pair 
matches  the  code  by  pressing  Yes  or  N. 

•  Assesses  visual  scanning  and  attention, 
learning,  and  immediate  recall. 


Spatial  Discrimination  (SPD) 

•  Pairs  of  four-bar  histograms  are  displayed 
on  the  screen  simultaneously,  and  the 
subject  is  requested  to  determine  whether 
they  are  identical. 

•  One  histogram  is  always  rotated  either 
±90  degrees  with  respect  to  the  other 
histogram. 

•  Assesses  visuospatial  analytic  ability. 


Sternberg  Memory  (STN) 

•  The  subject  memorizes  a  set  of  five 
letters,  after  which  letters  appear  on 
the  screen  one  at  a  time,  and  the 
subject  determines  if  the  letter  on 
the  screen  is  a  member  of  the 
memory  set. 

•  Assesses  working  memory. 


Results 


Table  1:  Characteristics  of 
study  sample  (n  =  6) 

Parameter 

%  or  Mean 

Female  (%) 

67 

Age  (years  old) 

43.2 

White  (%) 

100 

MMSE  vs  Sternberg  MS  Throughput 


PROCESSING  SPEED  SENSITIVE 

Reaction  Time  1 

RT1  Mean  RT  Correct  Responses 

RT1  Median  RT  Correct  Response 

RT1  Mean  RT 

RT1  Median  RT 

RT1  Throughput 

-0.7197 

-0.7432 

-0.7353 

-0.7109 

0.7051 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

Procedural  Reaction  Time 

PRT  Mean  RT  Correct  Responses 

PRT  Median  RT  Correct  Response 

PRT  Mean  RT 

PRT  Median  RT 

PRT  Throughput 

-0.7386 

-0.7085 

-0.7388 

-0.7092 

0.7098 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

Code  Substitution 

CS  Mean  RT  Correct  Responses 

CS  Median  RT  Correct  Responses 

CS  Mean  RT 

CS  Median  RT 

CS  Throughput 

-0.7603 

-0.7843 

-0.7467 

-0.7761 

0.7251 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

ACCURACY  SENSITIVE 

Sternberg  Memory  Search 

MS  Number  Correct 

MS  Percent  Correct 

MS  Number  Incorrect 

MS  Percent  Incorrect 

0.8122 

0.8122 

-0.8122 

-0.8122 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

All  Data  Presented: 

Spatial  Processing 

SP  Number  Correct 

SP  Percent  Correct 

SP  Number  Incorrect 

SP  Percent  Incorrect 

Parameter 

0.7897 

0.7897 

-0.7897 

-0.7897 

Pearson  r 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

P  value  (two-tailed) 

Code  Substitution 

CS  Number  Correct 

CS  Percent  Correct 

CS  Number  Incorrect 

CS  Percent  Incorrect 

0.7711 

0.7711 

-0.7711 

-0.7711 

<  0.0001 

<  0.0001 

<  0.0001 

<  0.0001 

MMSE  vs  Sternberg  MS  #  Correct  Responses 


MMSE  vs  Sternberg  MS  #  Incorrect  Responses 


Conclusion 


The  neurocognitive  tests  employed  in  the  DANA  mobile  assessment  were  highly 
correlated  with  conventional  MMSE  measurements  employed  in  the  treatment  of 
depressed  patients  undergoing  ECT  treatment.  Results  suggest  that  DANA  is  an  effective 
tool  for  assessing  cognition  in  patients  with  major  depression. 
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Appendix  H 


Abstract 

Post  deployment  stress  is  marked  by  high 
rates  of  post-traumatic  stress,  depression, 
insomnia,  and  other  mental  health 
conditions  that  may  go  undiagnosed  and 
therefore  untreated.  In  the  military,  there 
is  often  under-reporting  or  over-reporting 
of  symptoms  indicative  of  mental  or 
physical  health  problems  associated  with 
impaired  performance.  Assessments 
based  on  self-reported  symptoms  are  not 
always  effective  in  detecting  underlying 
psychological  conditions  or  problems.  This 
study  examined  the  potential  for  using 
computerized  neurocognitive  tests  (CNTs) 
as  an  objective  biomarker  of  functional 
impairment  in  a  functioning,  non-clinical, 
military  population  in  order  to 
complement  subjective  reporting  of 
symptoms.  Although,  extensive 
neuropsychological  testing  can  be 
performed  for  research  purposes,  this  is 
often  difficult  in  clinical  practice  as  there 
are  relatively  few  neuropsychologists; 
time  and  cost  is  often  prohibitive,  and 
traditional  measures  may  only  detect 
severe  impairment.  CNTs  using  precise 
speed  and  accuracy  measures  hold  the 
promise  of  being  more  sensitive,  as  well 
as  being  more  easily  disseminated  and 
utilized  in  a  variety  of  healthcare  settings. 
This  research  aimed  to  determine  if 
neurocognitive  functioning  was  associated 
with  emotional  and  somatic  distress  in  a 
sample  of  combat  veterans.  Differences 
for  those  with  and  without  significant 
symptoms  of  PTSD,  Depression,  Insomnia, 
and  Anger  were  compared  across  seven 
neurocognitive  tests.  Sensitivity  and 
specificity  of  the  tests  were  also 
determined.  Medium  to  small  effect  sizes 
were  found  on  cognitive  performance  for 
all  psychological  factors.  Results  suggest 
that  the  CNT  provided  a  reliable  and 
sensitive  measure  that  has  the  potential 
for  being  utilized  by  clinicians  for  initial 
diagnosis,  treatment  disposition,  and 
potentially  assessing  improvement  and 
return  to  work  determinations. 

Introduction 

Currently,  there  are  no  standardized, 
objective  assessments  or  biomarkers 
available  for  predicting  or  determining 
diagnostic  cut-offs  for  psychological 


conditions  or  problems  such  as  PTSD, 
Depression,  Insomnia,  Anger,  or  post- 
concussive  symptoms.  CNT  data  was 
collected  from  active  duty  personnel  using 
the  Defense  Automated  Neurobehavioral 
Assessment  (DANA).  DANA  is  a  CNT  that 
was  developed  for  the  Department  of 
Defense  (DoD)  as  a  clinical  decision 
support  tool  for  assessing 
neurobehavioral  functioning  both  in-field 
and  in-clinic.  This  research  investigates 
the  general  assumption  that  various  risk 
factors  (PTSD,  Depression,  Insomnia, 
anger,  pain,  etc.)  lead  to  significantly 
slower  and  less  accurate  (throughput) 
performance  on  cognitive  tests.  DANA's 
primary  purpose  is  to  assist  providers  in 
determining  current  level  of  cognitive 
functioning  and  to  track  recovery  over 
time.  This  study  analyzed  DANA's  ability 
to  operate  as  an  objective  neurocognitive 
biomarker  for  cognitive  impairment  that  is 
associated  with  PTSD,  Depression, 
Insomnia,  Anger,  and  post-concussive 
symptoms  (3+  or  0-1).  The  study  also 
determined  the  sensitivity  and  specificity 
of  DANA  neurocognitive  measures  to 
discriminate  the  accurate  diagnosis  for 
this  population. 

Question  1:  Do  CNT  tests  or 
composite  scores  correspond  to 
diagnostic  cut-offs  for  PTSD, 
Depression,  Insomnia,  Anger,  or 
post-concussive  symptoms  for  0- 
1  or  3  or  more  concussions? 

Question  2:  Which  CNT  tests  and 
composite  scores  are  most 
closely  associated  with  diagnostic 
cut-offs  for  PTSD,  Depression, 
Insomnia,  Anger,  or  post- 
concussive  symptoms  for  0-1  or  3 
or  more  concussions? 

Methods 

DANA  Standard  military  version  CNT  was 
administered  on  hand-held  devices  to  646 
active  duty  service  members  from  the  2nd 
Marine  Expeditionary  Force.  The  battery 
included  assessments  for  emotional 
distress  (PCL-m,  PHQ-8,  anger); 
postconcussive  symptoms  (based  on  a 
revised  NSI-22;  insomnia  (PSQI),  and 
neurocognitive  functioning  (Simple 
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Reaction  Time  -  SRT;  Procedural  Reaction 
Time  -  PRT);  Go-NoGo  (GNG),  Spatial 
Rotation  Discrimination  (SRD),  Code 
Substitution  Learning  (CDS),  Code 
Substitution  Delayed  (CDD),  and 
Sternberg  Running  Memory  (STN). 
Following  IRB-approved  consenting  of 
participants,  testing  took  about  40 
minutes.  Subjects  were  generally 
between  the  ages  of  19-22,  and  were 
either  Never  Deployed,  Previously 
Deployed  (more  than  one  year  prior  to 
testing),  or  Recently  Deployed  (within  six 
months  of  testing).  Throughput  (TP)  was 
calculated  for  each  neurocognitive 
measure  (number  of  correct  responses  in 
one  minute).  A  Simple  Composite  Score 
based  on  four  simple  reaction  time  tests 
was  calculated  by  averaging  the  z  scores 
of  SRT  TP;  PRT  TP;  GNG%  and  CDS  [(zSRT 
TP  +  zPRT  TP,+  zGNG%  +  zCDS)  /  4],  A 
Complex  Composite  Score  was  the 
average  of  the  z  scores  for  the  three  more 
cognitively  tasking  tests  SRD  TP;  CDD  %; 
and  STN%  [(zSRD  TP  +  zCDD  %  +  zSTN%  )  / 
3].  The  Total  Composite  Score  was  the 
average  of  the  two  composite  scores 
[(Simple  +  Complex)  /  2].  Cut-offs  and 
measures  for  each  symptom  were  as 
follows:  PTSD  (PCL-m  >49),  Depression 
(PHQ-8  >10),  Anger  (DSI  >1SD),  and 
insomnia  (PSQI>11). 

Results 

General  Linear  Modeling  was  used  for 
between-group  ANOVA  to  compare 
groups  above  and  below  the  diagnostic 
cut-off  for  each  symptom.  Generalized 
Linear  Modeling  (with  Wald  statistic  and 
Robust  Estimator)  was  utilized  to  confirm 
significance  when  groups  were  associated 
with  unequal  variances  per  Levene's  test 
of  homogeneity.  Medium  to  small  effect 
sizes  were  found  on  CNT  performance  for 
all  psychological  factors.  Receiver 
Operating  Characteristic  (ROC)  curve 
analyses  was  done  to  measure  sensitivity 
and  specificity  of  the  measures  for  the 
different  diagnostic  cut-offs.  See  table 
and  figures  for  details.  The  z  score  for  SRT 
TP  showed  significant  differences  for  all 
factors.  The  Simple  Composite  was  more 
sensitive  than  the  Complex  Composite 
Score  overall.  Details  of  the  findings  are 
presented. 
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Simple  Composite  Score  for  PTSD 
(F=  7.342,  P=.007,  q2p=.018) 


Complex  Composite  Score  for  PTSD 
(F=  10.878,  P=.001,  r|2p=.037) 


Total  Composite  Score  for  PTSD 
(F=  10.190,  P=.002,  ri2p=.036) 
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Simple  Reaction  Time  Throughput  for  ANGER  (F=11.795, 
P=.001,  ri2p=.018) 


Simple  Reaction  Time  Throughput  for  INSOMNIA 
(F=  6.207,  P=.013,  r|2p=.017) 


Simple  Reaction  Time  Throughput  for  DEPRESSION 
(F=  7.740,  P=.006,  r|2p=.018)  (Wald=5.458,  DF=1,  P=.019 


Jack  Tsao,  MD,  DPhil 
Bureau  of  Medicine  and  Surgery 
Department  of  Defense 


Sensivity  and  Specificity  of  zSRT_TP  and  Total 
Composite  Scores 


ROC  curves  for  >3  lifetime  concussions 
(Simple  and  Total  Composite  Scores 
were  significant  to  <.01) 


ROC  curves  for  >3  lifetime  concussions 
(Simple  and  Total  Composite  Scores 
were  significant  to  <.01) 
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ROC  curves  for  Insomnia  (ISI  >12) 
(zSRT_TP  was  significant  to  <.05) 


ROC  curves  for  PTSD  (PCL-M>49) 

(zSRT_TP,  Simple,  and  Total  significant  to  <.05) 


ROC  curves  for  Depression  (PHQ-8  >10) 
(zSRT_TP  significant  to  <.05) 


ROC  curves  for  Anger  (NSI  >1SD) 
(zSRT_TP  significant  to  <.05) 
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Background 


Sleep  deprivation  and  fatigue  are  major  contributors  to 
>erformance  degradation  in  laboratory  and  operational 
settings1-3.  In  continuously  operating  work  environments, 
>erformance  decrements  can  have  serious  consequences.  In 
act,  serious  accidents  and  fatalities  as  a  result  of  driving  while 
sleepy  are  well  documented4.  Evaluating  operational  readiness 
equires  understanding  of  available,  relevant  information  with 
he  means  to  detect  and  adjust  for  shortcomings.  In  studies  of 
>perational  performance,  additional,  and  often  simple,  tasks 
ire  also  examined  to  assess  neurocognitive  performance.  It 
:an  be  generalized  that  some  constructs  measured  by 
>perational  and  neurocognitive  tasks  overlap,  but  the 
elationship  and  utility  of  using  one  to  predict  another  under 
he  condition  of  sleep  deprivation  has  been  lacking. 

This  study  examined  the  relationship  between 
leurocognitive  and  driving  performance  during  50h  of  sleep 
leprivation  in  subjects  receiving  caffeine  or  placebo, 
’erformance  over  time  was  assessed  and  the  relationships 
>etween  performance  metrics  were  sought  in  an  effort  to 
>redict  operational  performance  using  a  neurocognitive  test. 


q  1A.  DANA  and  IB.  Military  Energy  Gum® 


Brtici pants 

•Twenty-four  healthy  subjects  (9  female)  aged  18-31(22.5±2.9) 
yr,  randomly  assigned  to  caffeine  (CAF,  n=12)  or  placebo  (PLA, 
n=12)  groups. 

•Low  to  moderate  CAF  users  (<250mg  CAF/day). 

•  Regular  sleep  wake  patterns  with  an  average  total  sleep  time  of 
8.6±0.6h,  verified  with  sleep  diaries  and  activity  monitors. 

easures 

Defense  Automated  Neurobehavioral  Assessment  (DANA)  - 

Rapid  The  DANA  (Fig  1A)  is  a  5  min  neurocognitive  battery 
consisting  of  three  subtasks:  simple  reaction  time  (SRT), 
procedural  reaction  time  (PRT)  and  a  go-no-go  (GNG)  task. 
Primary  variables  were  reaction  time  (RT),  accuracy  (ACC), 
Lapses  (LAPS)  and  throughput  (TP) 

Driving  Simulator  -  The  driving  task  simulated  a  monotonous 
drive  at  dusk  thru  the  Australian  outback  (little  scenery,  few 
road  cues)  and  the  requirement  to  maintain  speed  at  80  kph. 
Each  40  min  test  was  completed  on  a  full  motion  platform  with 
audio  ques. 

Primary  variables  were  speed  maintenance  (SM)  lane  deviations 
(LD)  and  crashes  (CR). 

esian/ Procedure 

Participants  had  baseline  sleep  (lOh  TIB,  Day  1/2),  followed  by 
50h  of  sleep  deprivation  (beginning  Day  2  at  0800)  and  one 
daytime  recovery  sleep  period  (9h  TIB,  Day  5). 

Participants  chewed  two  pieces  (100  mg  CAF/piece)  of  Military 
Energy  Gum®  (Marketright  Inc,  Plano,  IL,  Fig  IB)  or  PLA  at 
0100,  0300,  0500,  and  0700  on  Days  3  &  4  during  sleep 
deprivation. 

Throughout  the  study  participants  completed  a  range  of 
neurobehavioral  tests  every  4  h. 

Between  test  periods  participants  refrained  from  physical 
activity  (casual  walking  was  allowed  if  participants  were 
struggling  to  stay  awake),  but  could  do  leisure  activities  (e.g., 
watch  DVD's,  play  board/card  games,  etc.)  in  a  shared  lounge 
area. 


Statistical  A  na/vses 

Mixed-model  analysis  (3-way)  was  used  to  analyze  between  group  differences  and  change  over  time 
for  DANA  and  driving  performance  (DRP).  Fixed  effects  included  group  (CAF,  PLA),  gender,  and  time 
(15  test  periods)  with  a  random  effect  for  subjects.  Between-subject  correlations  (Pearson  product- 
moment)  were  used  to  assess  the  relationship  between  average  DANA  and  DRP  variables  over  time  by 
correlating  mean  performance  variables  (by  group)  at  each  of  the  15  test  periods.  CAF  &  PLA 
correlations  that  were  largest  and  similar  in  magnitude  were  used  to  identify  the  DANA  metric  that  had 
the  strongest  relationship  with  driving  performance  (DANA  RT,  LAPS  &  TP;  DRP  LD  &  CR).  Within- 
subject  correlations  (Pearson  product-moment)  were  calculated  for  each  individual  over  time  to  assess 
DANA  and  associated  DRP  change  within  each  individual  and  within-subject  correlations  were 
summarized  by  direction  and  significance.  A  'common'  within-subject  correlation  for  each  group  was 
calculated  using  analysis  of  covariance;  largest  PLA  correlations  were  used  to  identify  the  strongest 
DANA  and  driving  performance  relationship.  Lapse  and  crash  data  were  analyzed  after  applying  a 
Logl0(x+1)  transformation.  Statistical  significance  was  set  at  p  =  0.05. 


Results 


•There  were  significant  declines  in  all  DANA  subtask  metrics  and  DRP  metrics  (time  [p  <  .0001]  and 
group  x  time  interaction  [p  c.001]). 

•  Significant  differences  from  baseline  in  DANA  and  DRP  were  observed  at  similar  times  (0300-0600, 
Day3)  for  PLA  for  all  performance  measures  (Fig  2B,  3B). 

•CAF  performance  declined  with  an  increase  in  LD  and  SRT  TP  after  0300  Day4  (Fig  2A)  while 
performance  was  maintained  in  CR  and  SRT  LAPS  (Fig  3A). 

•  Correlations  for  24  pairs  of  DANA  and  DRP  data  were  analyzed  for  each  group. 

•All  CAF  and  PLA  between-subject  correlations  were  significant. 

•  Strongest  between-subiect  correlations  (Table  1):  LD  and  SRT  TP  (Fig  4A);  CR  and  SRT  LAPS  (Fig  4B). 
•Within-subject  correlations  showed  directional  coherence  in  82%  -  100%  of  subjects  and  35%  - 
75%  were  significant. 

•All  CAF  and  PLA  common  within-subject  correlations  were  significant  except  CAF  CR  and  PRT  LAPS 
correlation. 

•Strongest  within-subject  correlations:  LD  and  SRT  TP  (rcaffeine  =  -.60,  pc. 0001;  rp|acebo  =  -.71, 
pc. 0001);  CR  and  PRT  TP  (rcaffeine  =  -.32,  pc.0001;  rplacebo  =  -.64,  pc.0001). 


Fig  2.  Group  Means  Over  Time  in  Lane  Deviations  and  SRT  Throughput  for  CAF  (A^  and  PLA  (By  Red  arrows 

indicate  CAF/ PLA  administration  times. 


Fig  3.  Group  Means  Over  Time  in  Crashes  and  SRT  Lapses  for  CAF  (A^  and  PLA  (B^.  Red  arrows  indicate 

CAF/ PLA  administration  times. 


Fig  4.  Between-subiect  correlation  plots  for  Lane  Deviations  and  SRT 

Throughput  (A^  and  Crashes  and  SRT  Lapses  (B^.  Red  circles  are  mean  CAF 

performance  and  blue  squares  are  mean  PLA  performance. 


Table  1.  Strongest  between-subiect  (and  corresponding  within-subiecf) 

correlations  for  Driving  and  DANA  performance  in  CAF  and  PLA  groups. 

Additional  correlations  between  DP  and  DANA  metric  are  available  upon 
request.  *pc.05 


Lane  Deviations  &  SRT 
Throughput 

Crashes  &  SRT  Lapses 

Between-subject  Within-subject 

Between-subject 

Within-subjec 

Caffeine 

-0.97* 

-0.60* 

0.88* 

0.21* 

Placebo 

-0.96* 

-0.71* 

0.89* 

0.59* 

Summary 


•The  onset  of  DANA  and  driving  performance  decrements  appeared  t( 
be  similarly  impacted  by  sleep  deprivation  after  0300  (20  hr  awake)  ir 
the  placebo  group  and  end  of  the  study  in  the  caffeine  group. 

•The  strength  of  the  between-subject  correlations  demonstrate  that  th< 
DANA  can  be  used  as  an  indicator  of  average  driving  (i.e.,  operational 
performance. 

•DANA  SRT  had  the  strongest  relationships  with  driving  measures  ir 
both  groups. 

•  Lane  deviations  had  the  strongest  between-  and  within-subjec 
correlations  with  SRT  throughput:  decreases  in  throughput  (correc 
responses  rate  with  respect  to  reaction  time)  was  related  to  increase: 
in  driver  swerving. 

•Crashes  had  the  strongest  between-subject  relationship  with  SRI 
lapses:  as  the  number  of  lapses  increased,  so  did  the  average  numbe 
of  crashes.  Within-subject  correlations  for  crashes  had  a  stronge 
relationship  with  PRT  throughput. 

•The  magnitude  and  directional  coherence  of  within-subject  correlation: 
support  that  DANA  and  driving  performance  relationships  hold  true  foi 
an  individual. 

•Within-subject  correlations  were  not  as  strong  as  between -subjec 
correlations  most  likely  a  result  of  interindividual  difference  (e.g. 
sensitivity/ insensitivity  to  caffeine/ sleep  loss,  etc.). 

•  Caffeine  within-subject  correlations  with  crashes  were  low  because  th< 
majority  of  subjects  given  caffeine  did  not  crash. 

•Overall,  the  consistency,  magnitude,  and  statistical  significance  o 
between-  and  within-subject  correlations  demonstrates  the  aptitude  o 
DANA's  ability  to  predict  driving  performance  (average  and  individual 
during  sleep  deprivation. 
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Differences  in  Cognitive  Efficiency  in  Active  Duty  Military  -  Healthy  vs.  Those  With  Sleep  Disturbance,  Depression, 
and/or  PTSD  Using  The  DANA  Automated  Neurocognitive  Assessment  (DANA)  Tool 

Corinna  Lathan,  Angela  S.  Wallace,  and  Rita  Shewbridge 

AnthroTronix,  Inc. 


Introduction 

Neurocognitive  assessment  tools  (NCATs)  are  key  to  assessing  neurocognitive  function  quickly  and 
effectively  in  operational  environments.  The  Defense  Automated  Neurobehavioral  Assessment  (DANA)  is  a 
portable  FDA-cleared  NCAT.  DANA  has  demonstrated  excellent  reliability  in  a  variety  of  military  field 

conditions.  . 

Methods 

Data  Collection:  DANA  was  administered  to  806  active-duty  military  (71%  Male)  ages  18  to  64. 

Measures:  Simple  Reaction  Time  (RT)  -  Participant  taps  location  of  yellow  target  as  quickly  as  possible 
each  time  it  appears.  Procedural  Reaction  Time  ( PRT )  -  Screen  displays  one  of  four  numbers  for  three 
seconds.  Subject  presses  on  left  button  (“2”  or  “3”)  or  right  button  (“3”  or  “4”)  depending  on  number 
presented.  Code  Substitution  Learning  (CSL)  -  Subject  refers  to  a  code  of  9  symbol-digit  pairs  shown 
across  the  upper  portion  of  the  screen.  A  sequence  of  single  symbol-digit  pairs  is  shown  below  the  key. 
Subject  indicates  whether  or  not  the  single  pair  matches  the  code  by  pressing  “Yes”  or  “No”. 

Go/No-Go  ( GNG )  -  A  house  is  presented  on  screen  with  several  windows.  Either  a  “friend”  (green)  or 
“foe”  (white)  appears  in  a  window.  Subject  must  push  a  “fire”  button  only  when  “foe”  appears. 


Simple  Reaction  Time  (RT) 


Procedural  Reaction  Time  (PRT) 


Data  Analysis:  T-tests  comparing  mean  throughput  (a  measure  of  cognitive  efficiency;  Thorne,  2006)  on 
AM  and  PM  administrations  and  mean  throughput  between  self-reported  healthy  and  “unhealthy” 
participants  (i.e.  reported  symptoms  related  to  sleep  disturbances  [PSQI  >  7],  depression  [PHQ8  >  10], 
and/or  PTSD  [PCLM  >  30])  was  conducted. 

Results:  Diurnal  Analysis 

AM  vs.  PM  Administrations 


RTl  CSL1  PRT1  SP1  GNG1  CSD2  MTS1  RT2 

[■AM  184.0  43.7  93.2  32.4  112.1  j  45.6  33.9  175.1 

|  "PM  |  184.9  1  43.4  [  91.8  1  32.2  |  114.0  |  45.7  |  33.6  |  176.0 

Nm  =  360-424  WPM  =  350-406.  *p  <  .05,  **p  <  .01 ,  ***p  <  .001 . 

No  significant  differences  in  mean  throughput  between  AM  and  PM  for  any  DANA  subtests. 


Results:  Optimal  Well-Being  versus  Compromised  Well-Being  by  Condition 

Combat  Exposure 


Healthy 


PTSD  Only 


Sleep  Only 


Sleep  and  PTSD  Dep,  Sleep,  and 
PTSD 


Differences  in  Combat  Exposure:  Healthy  participants  had  the  least  amount  of  combat  exposure 
[M  =  4.25)  followed  closely  by  participants  with  disturbed  sleep  [M  =  4.30).  The  level  of  combat  exposure 
experienced  by  these  two  groups  is  considered  “light  combat”  (Keane  et  al.,  1989).  Participants  who 
reported  PTSD  experienced  “light  to  moderate  combat”  exposure.  Of  those  in  groups  reporting  PTSD, 
participants  with  only  PTSD  had  the  least  combat  exposure  (M  =  9.71)  followed  by  those  with  depression, 
disturbed  sleep,  and  PTSD  ( M  =  9.83).  Finally,  participants  with  disturbed  sleep  and  PTSD  had  the  most 
combat  exposure  (M  =  11 .77). 

Healthy  vs.  PTSD  ONLY 


U - 

RTl 

CSL1+ 

PRT1* 

GNG1 

RT2* 

■  Healthy 

185.7 

44.1 

93.5 

113.9 

178.6 

■  PTSD  ONLY 

183.3 

41.9 

90.3 

113.8 

172.8 

Healthy  “  521-552,  WpTSDONLY  -  66-67.  *p  <  .05,  **p  <  .01,  ***p  <  .001 . 

Healthy  participants  have  significantly  higher  PRT1  and  RT2  mean  throughput  than  participants  with 
PTSD. 


Healthy  vs.  Sleep  Disturbed  ONLY 


u  — 

RTl 

CSL1 

PRT1 

GNG1 

RT2* 

■  Healthy 

185.7 

44.1 

93.5 

113.9 

178.6 

■  Sleep  Disturbed  ONLY 

182.7 

43.3 

92.0 

111.3 

170.6 

^Healthy -  521  -552,  A/S,eep  Disturbed  0nly  -  95-1 02.  *p  <  .05,  **p  <  .01 ,  ***p  <  .001 . 

Healthy  participants  have  significantly  higher  RT2  mean  throughput  than  sleep  disturbed  participants. 


Results:  Optimal  Well-Being  versus  Compromised  Well-Being  by  Condition  (cont’d) 

Healthy  vs.  Sleep  Disturbed  and  PTSD 


u 

RTl 

CSL1* 

PRT1+ 

GNG1 

RT2 

■  Healthy 

185.7 

44.1 

93.5 

113.9 

178.6 

■  Sleep  and  PTSD 

182.2 

41.7 

88.6 

110.5 

168.3 

^Healthy  =  521  -552,  A/Sleep  and  PTSD  =  36-37.  *p  <  .05,  **p  <  .01 ,  ***p  <  .001 . 

Healthy  participants  have  significantly  higher  CSL1  mean  throughput  than  participants  with  disturbed  sleep 
and  PTSD. 

Healthy  vs.  Sleep  Disturbed,  Depressed,  and  PTSD 


u  - 

RTl** 

CSL1** 

PRT1** 

GNG1** 

RT2*** 

■  Healthy 

185.7 

44.1 

93.5 

113.9 

178.6 

■  Sleep,  Depressed,  and  PTSD 

172.1 

40.3 

87.4 

104.6 

154.8 

'Healthy  =  521-552,  WS|eep>  Depressed,  and  PTSD  0NLY  =  36-41.  *p  <  .05,  **p  <  .01,  ***p  <  .001. 

Healthy  participants  have  significantly  higher  mean  throughput  on  RTl ,  RT2,  CSL1 ,  PRT  1 ,  and  GNG1  than 
participants  with  all  three  conditions  combined. 

Conclusions 

First,  DANA  provides  stable  assessments  of  mean  throughput  regardless  of  the  time  of  day  DANA  is 
administered.  Second,  active  duty  military  who  report  being  depressed,  having  disturbed  sleep,  and/or 
PTSD  have  worse  cognitive  performance  on  reaction  time,  simple  executive  functioning,  attention,  and 
speed  and  accuracy  of  targets  than  active  military  who  do  not  report  any  of  the  aforementioned  conditions. 
Lower  cognitive  efficiency  can  have  potentially  harmful  effects  to  individuals  and  to  others  with  whom  these 
individuals  interact  with  to  complete  various  activities  while  in  the  field  (e.g.  responding  to  a  target  in  time 
to  thwart  an  attack).  DANA  can  aid  in  identifying  decreases  in  cognitive  efficiency  in  the  field. 
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Title:  Longitudinal  evaluation  using  the  recently  developed  Defense 
Automated  Neurobehavioral  Assessment  (DANA)  tool  of  the  cognitive  impact  of 
Electroconvulsive  Therapy  (ECT)  in  the  treatment  of  Major  Depressive  Disorder  (MDD) 
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Spring  MD 

Major  Depressive  Disorder  (MDD)  has  been  a  major  source  of  morbidity  and 
mortality,  and  is  often  associated  with  cognitive  impairment.  Little  is  understood  about 
the  cognitive  effects  of  electroconvulsive  therapy  (ECT)  during  the  treatment  course 
among  patients  with  MDD.  Our  study  represents  a  novel,  automated  assessment  of  the 
effect  on  cognitive  functioning  of  patients  receiving  ECT  treatment  for  MDD.  The 
objectives  of  this  study  were  to  measure  cognitive  changes  in  MDD  patients  during 
their  course  of  ECT  treatment.  We  also  correlated  cognitive  measures  obtained  by 
administering  DANA  with  results  of  mini  mental  status  examinations  (MMSE).  DANA 
is  a  new  neurocognitive  assessment  tool  that  includes  a  library  of  standardized 
cognitive  and  psychological  assessments.  A  neurocognitive  and  psychological 
assessment  battery  was  conducted  on  an  electronic  tablet  with  DANA  software, 
throughout  the  course  of  ECT  treatment.  Results  of  DANA  assessments  were  then 
correlated  with  standardized  MMSE  results  employed  for  clinical  purposes.  Preliminary 
analyses  of  pilot  data  indicate  that  DANA  performance  correlated  with  MMSE  in  the 
majority  of  outcomes.  The  correlation  between  MMSE  and  aspects  of  the  following 
tests  gave  an  absolute  correlation  coefficient  r  >  0.7,  with  p<0.0001:  Code  Substitution, 
Spatial  Processing,  Procedural  Reaction  and  Simple  Reaction  Time.  Correlation 
between  MMSE  and  aspects  of  the  Sternberg  Memory  Search  gave  an  even  higher 
absolute  correlation  coefficient  r  >  0.8,  p<0.0001.  The  neurocognitive  tests  employed 
in  the  DANA  mobile  assessment  were  highly  correlated  with  conventional  MMSE 
measurements  employed  in  the  treatment  of  depressed  patients  undergoing  ECT 
treatment.  Results  suggest  that  DANA  is  an  effective  tool  for  assessing  cognition  in 
patients  with  major  depression. 
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Sciences.  His  research  interests  include  studying  cognition  among  patients  with  major 
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Background 

Feasibility  and  Reliability  Data 

Validity  and  Utility  -  Cross-Sectional  Data 

Longitudinal  Validity  -  Sensitivity  to  Impairment 
over  time 

Clinical  Studies 
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History  of  DANA 


& 


Developed  for  the  Department  of  Navy  Bureau  of  Medicine  and 
Surgery  (BUMED)  in  response  to  RFQ 

■  A  mobile  software  application  to  assist  in  the  detection  of 
neurocognitive  impairment  from  any  cause 

Concussion 


Combat-related  psychological  distress 
•  Deployment-related  exhaustion 

■  To  help  the  far  forward  medic  to  more  accurately  detect 
impairment  as  soon  as  possible 

■  To  assist  the  psychologist  or  healthcare  professional  with 
supportive  diagnostic  tools  to  aid  in  their  diagnosis 
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DANA  Requirements 


& 


•  Portability 

•  State  of  the  art  NeuroCognitive  Assessment  Tool  (NCAT) 
-  Build  on  Best  Practices 

•  Both  standardized  cognitive  and  psychological  tests 

•  Embedded  tests  of  effort  determine  malingering  vs  real 

•  Easy  of  use  for  all  types  of  providers  -  3  levels  of  testing 

•  FDA  Compliance 

•  Non-Proprietary 
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Platform 


•  Android  OS  (Beta  iOS) 

•  Java  Implementation 

•  Stylus  or  Touch  Screen 
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4 


DANA  Tests 


■ 


DANA  Rapid 

•  within  24  hours  following  concussion 

•  to  support  corpsman's/medic’s  disposition 

DANA  Brief 

•  follow  up  to  concussion  assessment  (after  24  hours) 

•  or  when  the  Medic  suspects  impairment  from  any  cause 
(concussion,  emotional  trauma,  fatigue) 

’DANA  Standard 

•  for  use  by  MO,  BH,  or  allied  healthcare  professional  to 
support  diagnosis  and  disposition 

•  to  assist  in  report  writing  and  EMR 


Human  Inspired.  Technology  Driven. 


DANA  Test  Batteries 


DANA  Rapid 
(5  minutes) 

DANA  Brief 
(15  minutes) 

DANA  Standard 
(45  minutes) 

Simple  Reaction  Time 

Simple  Reaction  Time 

Simple  Reaction  Time 

Procedural  Reaction  Time 

Code  Substitution  (Learning) 

Code  Substitution  (Learning) 

Go/No-Go 

Procedural  Reaction  Time 

Procedural  Reaction  Time 

Optional:  Combat  MACE 
interview  (additional  10 
minutes) 

Spatial  Processing 

Spatial  Processing 

Go/No-Go 

Go/No-Go 

Code  Substitution  (Recall) 

Code  Substitution  (Recall) 

Simple  Reaction  Time 

Matching  to  Sample 

Patient  Health  Questionnaire 
(PHQ-9) 

Sternberg  Memory  Search 

Primary  Care-PTSD  Screen 
(PC-PTSD) 

Simple  Reaction  Time 

Insomnia  Severity  Index  (ISI) 

Combat  Exposure  Scale  (CES) 

Patient  Health  Questionnaire  (PHQ-9) 

Pittsburg  Sleep  Quality  Index  (PSQI) 

PTSD  Check  List-Military  Version  (PCL-n 

Deployment  Stress  Inventory  (DSI) 

& 
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Output  Screens _ 

Tailored  for  use  by  a  range  of  provider  types 


Medic  MO/BH 


Neuropsychologist 

Neurologist 


DANA  Rapid  (NO  S) 

07/07/2011  at  19:20 


Administered : 
Given  to: 


by  Examiner 
John  Doe 


Overall  Results 


▲ 


Details n 

Normative  Results  For  Cognitive 
Tests 

Test  Percentile 

Reaction 
Time 

Procedural 
Reaction 
Time 
Go  No  Go 


1  Report  Sections  1 

Test 

Results 

1 

CES 

Survey 

Functioning:  Normal 
range. 

Overall  Score:  0 

Combat  Exposure 
(CES)  Light 

PHQ 

Survey 

Functioning:  Normal 
range. 

Overall  Score:  0 

PSQI 

Survey 

Functioning:  Normal 
range. 

Overall  Score:  0 

PCLM 

Survey 

Functioning: 

Abnormal  range. 

Overall  Score:  57 

Score:  4  -  Re- 
Experiencing 

Score:  22  -  Avoidance 
Score:  20  -  Hyper- 
Arousal 

Report  Sections 


Trial  Sequence 


Key 

Incorrect 
Trial 

Fast  Trial 

Skipped 
Trial 

Details 


Correct 

Trial 

•  Lapsed 
'Trial 

Normalized 
I  Reaction 
Time 


V 


Trial 


Response 


Response 

Time 


Inter- 

Trial 

Interval 


FDA  Regulatory  Process 


& 


•  July  201 1  -  Draft  Guidelines  for  Mobile  Medical 
Applications  (MMA) 

•  April  2013-  Filed  513(g)  for  FDA  device 
classification 

•  September  201 3  -  Met  with  FDA 

•  September  2013  -  Final  MMA  Guidelines 

•  March  2014  -  Received  513(g)  response 

•  Implemented  Quality  System  processes  per  FDA 
21  CFR  Part  820 

•  July  2014  -  Filed  510(k)  for  premarket  notification 
and  authorization 

•  October  2014  FDA  Clearance  Received 
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FDA  Cleared  IFU 


& 


DANA  provides  clinicians  with  objective  measurements  of  reaction  time 
(speed  and  accuracy)  to  aid  in  the  assessment  of  an  individual’s 
medical  or  psychological  state. 

Factors  that  may  affect  the  measurement  of  reaction  time  include,  but  are 
not  limited  to  concussion,  head  injury,  insomnia,  post  traumatic  stress 
disorder  (PTSD),  depression,  attention  deficit  hyperactivity  disorder 
(ADHD),  memory  impairment,  dementia,  delirium,  prescription  and 
non-prescription  medication,  some  nutritional  supplements,  as  well  as 
a  variety  of  psychological  states  (e.g.  fatigue  and  stress). 

DANA  also  delivers  and  scores  standardized  psychological 

questionnaires.  DANA  results  should  be  interpreted  only  by  qualified 
professionals. 
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Scaleable  Software 


Brain  Health  Screening  with  DANA™  Software 

FAST 

Results  in  5  minutes 


COMPLETE 

Full  Spectrum  of  Tests  from  5-Minute 
Rapid  to  15-Minute  Brief  and  45-Minute 
Standard  Testing  Protocols 


ACCURATE 

Complements  patient  feedback  with 
objective  tests  of  cognitive  efficiency 
and  self-administered  questionnaires 


DANA  Rapid  -  5  Minutes 

Within  24  hours  of  symptom  onset. 
Measures  cognitive  function  / 
brain  vital  signs. 


DANA  Brief  -  15  Minutes 

Follow-up  to  initial  symptom 
assessment  (after  24  Hours) 
For  use  when  impairment  of 
cognitive  function  is  suspected, 


DANA  Standard  -45  Minutes 

For  use  by  physician  or  psychologist. 
Assists  with  reports  and  EMHR. 
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18-21;  Fort  Lauderdale,  FL. 

Kaplin  A,  Woods  S.  Longitudinal  evaluation  using  the  recently  developed 
Defense  Automated  Neurobehavioral  Assessment  (DANA)  tool  of  the 
cognitive  impact  of  Electroconvulsive  Therapy  (ECT)  in  the  treatment  of 
Major  Depressive  Disorder  (MDD).  Poster  session  to  appear  at:  Military 
Health  System  Research  Symposium.  2014  Aug  18-21;  Fort  Lauderdale, 
FL. 


Eonta  S,  Kamimori  G.  Impact  of  caffeine  on  neurocognitive  performance 
during  sleep  deprivation  using  the  Defense  Automated  Neurobehavioral 
Assessment  (DANA).  Poster  to  appear  at:  Military  Health  System  Research 
Symposium.  2014  Aug  18-21;  Fort  Lauderdale,  FL. 
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Validity  and  Utility  -  Cross-Sectional  Data 

Longitudinal  Validity  -  Sensitivity  to  Impairment 
over  time 
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Psychometric  Properties  of  DANA  (MilMed  2014) 


Five  extreme  deployable  settings  for  durability  and  technical  validation 

(n=240,  >40/site) 

•  Shipboard  (Yokosuka,  heavy  seas) 

•  Arctic  (Greenland,  winter) 

•  Desert  (29  Palms,  summer) 

•  Mountainous  (Bridgeport) 

•  Jungle  (Okinawa  Humid  Summer) 

Outcomes 

Robustness  of  technology  -  adequate  session  completion  and  data  collection 

User  experience/interface  feedback 

Acquired  distributions  of  data  for  power  estimates  for  subsequent  clinical  data 
collection  (total  combined  population,  assessed  by  environment,  age,  and  gender) 

Reliability  Measures 

-  Test-Retest  Reliability  is  adequate  (>.64  across  all  environments) 

-  Split-Half,  Odd-Even  Reliability  (.76-. 95) 

-  Compares  favorably  to  ANAM  on  similar  tests: 

-  DANA-ANAM  comparison(N=40):  DANA  had  superior  test-retest  r 

-  3  largest  ANAM  samples  published:  DANA  had  similar  mean/SD  ratios 
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Longitudinal  stability  and  reliability 

(Applied.  Psychol.  Measurement,  2015) 


Test-retest  reliability  of  the  DANA  Rapid  test  battery,  as 
administered  to  a  homogeneous  population  of  US  Air  Force 
Academy  football  team  players  before  the  commencement  of  the 
season,  during  the  season,  and  following  the  end  of  the  season. 

•  No  significant  differences  in  Throughput  (TP)  between 
testing  sessions  pre-  and  mid  season,  and  between  mid 
and  post-season  (p>0.05),  per  subtest. 

•  DANA  is  stable  across  the  time  scales  of  test  and  retest 
administrations. 
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Comparison  to  IMPACT  and  ANAM 


& 


DANA’s  ICC  correlation  coefficient  (R)  of  0.81  for  Reaction  Time  (RT) 
exceeds  that  of  both  ANAM4  (0.60)  and  ImPACT  (0.53)  and  DANA’s  ICC 
of  0.75  for  Procedural  Reaction  Time  (PRT)  exceeds  that  of  ANAM4 
(0.51). 


Battery 

Subtest 

<TP> 

SD  <  TP  > 

N 

ICC  (R) 

T1  T2 

T1  T2 

LB  UB 

DANA 

PRT 

102.9  106.8 

13.02  12.57 

89 

0.75 

0.60  0.84 

RT 

192.4  198.3 

26.77  23.54 

87 

0.81 

0.69  0.87 

ANAM4 

PRT 

97.62 

13.10  10.31 

50 

0.51 

103.56 

0.28  0.69 

RT 

86.88  91.38 

11.46  12.76 

50 

0.60 

0.39  0.75 

ImPACT 

RT 

0.60  0.61 

0.08  0.09 

44 

0.53 

0.28  0.71 
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Establishing  a  Normative  Data  Sample  in  a  Military  Population 
Ft.  Hood  Military  Base 


Active-duty  military,  aged  18-64,  provides  a  relevant  range  of 
subjects  by  which  a  normative  reference  base  can  be  used  for 
the  purpose  of  neurocognitive  test  comparison  when  baseline 
information  is  unavailable. 

•  Testing  session  consists  of: 

DANA  Standard  Test 

Psychological  screening  questionnaires 

Demographics 

Survey  of  concussion  history 

•  N  >  800  subjects 

•  Data  collection  is  ongoing 
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Establishing  a  Normative  Data  Sample  in  a  Military  Population 

Ft.  Hood  Military  Base 


Age  Bins  by  Gender 


Bin 

18-19 

20-24 

25-29 

30-34 

35-44 

45-54 

55-64 

M 

101 

100 

100 

98 

92 

58 

29 

F 

23 

93 

52 

29 

23 

13 

3 

Psychological  Characteristics  of  the 

Population 


PTSD 

None 

82% 

Mild  to 
moderate 

15% 

Severe 

3% 

Depression 

None 

94% 

Mild  to 
moderate 

4% 

Severe 

2% 

Insomnia 

None 

73% 

Mild  to 
moderate 

23% 

Severe 

4% 
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Cross-Sectional  Comparisons 


Can  an  NCAT  detect  invisible  wounds  of  war  in 
Service  Members  returning  from  combat 
deployment? 

How  does  Cognitive  Testing  (Throughput  scores) 
relate  to  Concussion;  PTSD;  Depression;  anger, 
pain,  Insomnia;  Postconcussive  Sx, 
deployment? 
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Methods 
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N  =  646  USMC  2MEF 
Deployment: 

-  Never  -  Prior  (>6  mo)  -  Recent  (<6  mo) 
Concussions  (self-report): 

-  Recency  (<6  mo)  -  Lifetime  number 
Cognitive  Measures 

-  DANA  Standard  (435) 

-  DANA  Rapid  (646) 

Psychological  Factors  (meeting  criteria  for): 

-  PTSD 

-  Depression 

-  Insomnia 

-  PsychoPhysical  Sx 

-  CES 
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Results  (J.  Neurotrauma  2014) 
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Neurocognitive  tasks  were  positive  for  distinguishing  between 
those  who  met  criteria  for  PTSD,  depression,  insomnia,  anger, 
pain,  and  postconcussive  symptoms  (NSI-22).  (via  ANOVA, 
and  Sensitivity/Specificity  analyses  (Moderate-Large  effect 
sizes) 

Cognitive  tests  could  distinguish  those  who  had  been  previously 
concussed,  but  only  in  those  who  had  three  or  more  lifetime 
concussions,  even  when  accounting  for  independent 
covariates  of  PTSD,  Depression,  deployment,  or  combat 
experiences. 

Only  simple  tests  (SRT,  PRT,  GNG,  CDS)  were  significant  for 
detecting  differences  between  normal  and  affected  groups 

-  suggesting  that  white  matter  functioning  is  being  tapped  into,  rather 
than  higher  cognitive  factors. 
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Background 

Feasibility  and  Reliability  Data 

Validity  and  Utility  -  Cross-Sectional  Data 

Longitudinal  Validity  -  Sensitivity  to 
Impairment  over  time 

*  High  Altitude  Hypoxia 

*  In-Theatre  Concussion 

Clinical  Studies 
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High  Altitude  Cognitive  Testing 


21  subjects  (12  males,  nine  females,  average  age  20.8  yrs, 
ranae  1 9-23  vrs) 


SL  ALT1  ALT7  ALT16  Post7/21 

5260 


^  3500 

0) 

-c 

D 

•j- 
< 

1500 

Sea  Level 

Time 


numcm  nib|jiit;u.  ieu  n  luiuyy  uiivcii. 


24 


Decreased  reaction  time  after  high  altitude 

cognitive  testing  is  a  sensitive  metric  of  hypoxic  impairment 


^ 
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Neurocognitive  Function  During  Acclimatization 
and  Upon  Reascent 
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In-Theater  Concussion  Assessment 

Camp  Leatherneck,  Afghanistan 


Cognitive  testing  with  DANA  and  ANAM  was  simultaneously 
administered  to  both  service  members  diagnosed  with  mild 
concussion  (n~17)  and  to  non-concussed  service  members 
(N~17). 

•  Testing  occurred  within  3  days  post-injury  (or  post-baseline  for  controls),  and  again  5 
days  post-injury. 


Results  (note  that  it  is  a  small  n  and  may  not  be  generalizable) 

•  DANA  and  ANAM  For  equivalent  tests,  DANA  and  ANAM  have  equivalent  sensitivity  (.93) 
to  concussion  and  recovery.  However  DANA  seems  to  have  a  higher  specificity  (.89), 
the  ability  to  reject  false  concussion,  than  ANAM  (.44) 

•  DANA  and  MACE  (n=40)  DANA  shows  concurrent  validity  against  MACE  in  that  DANA 
memory  tests  correspond  to  MACE  recall  tasks. 

•  DANA  Psychological  Assessment  Symptomatic  insomnia,  depression,  and  PTSD  are  all 
observed  to  greater  effect  in  concussed  patients  than  in  non-concussed  patients. 
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Estimated  Marginal  Means 


In-Theater  Concussion  Assessment 

Camp  Leatherneck,  Afghanistan 


•  DANA  showed  improvement  in  <TP>  over  time  with  no  difference  observed 
at  Time  3  in  concussed  vs.  controls. 

•  ANAM  showed  no  improvement  of  concussed  relative  to  non-concussed. 

•  Although  the  sample  size  was  small,  it  seems  that  in  this  case  DANA  was 
more  sensitive  to  changes  over  time  for  these  concussed  vs  non-concussed 
patients. 


Estimated  Marginal  Means  of  MEASURE_1  Estimated  Marginal  Means  of  MEASURE_1 
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Validity  and  Utility  -  Cross-Sectional  Data 

Longitudinal  Validity  -  Sensitivity  to  Impairment 
over  time 
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U.S.  Army  Rapid  Innovation  Fund 


PTSD  Study 

•  VA  Pacific  Island  Healthcare  System,  Dr.  Jim  Spira  (PHREI) 

•  100  PTSD  patients,  receiving  traditional  (in  clinic)  vs.  telehealth  treatment 

•  Using  DANA  to  track  recovery  over  time 

Concussion  Study 

•  University  of  Wisconsin,  (Dr.  Alison  Brooks) 

•  66  student  athletes  (football),  33  concussed,  33  matched  controls 

•  To  assess  cognitive  function  during  recovery  from  moderate — severe  TBI 
using  the  DANA 

Depression  Study 

•  Johns  Hopkins  University  Hospital,  Dr.  Adam  Kaplin 

•  40  clinically  depressed  patients,  ECT  vs.  non-ECT  treatment 

•  Evaluation  of  the  Cognitive  and  Behavioral  Impact  of  ECT  on  the 
Treatment  of  Major  Depressive  Disorder 
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Evaluation  of  the  cognitive  and  behavioral  impact  of  ECT  on  the 
treatment  of  major  depressive  disorder 


Major  Depressive  Disorder  (MDD) 

A  major  source  of  morbidity  and  mortality  in  active  military  personnel  and 
veterans  often  associated  with  cognitive  impairment.  Little  is  understood 
about  the  on-going  cognitive  effects  of  ECT  on  patients  with  MDD. 

Study  Objective 

To  measure  cognitive  changes  in  MDD  patients  during  their  course  of  ECT 
treatment  (3  weeks) 

Protocol 

•  Cognitive  Assessments:  twice  a  week 

•  Mini-Mental  Status  Exam  (MMSE):  2-3  times  per  week 

•  Subjects  n=20  ECT  subjects,  with  n=20  control  subjects  undergoing  other  types 
of  therapies 

•  Current  Progress:  8  ECT,  14  Controls 
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Results 


& 


•  There  is  a  high  correlation  between  DANA  and  MMSE 
Results.  (MHSRS  Poster  2014) 

•  However  MMSE  shows  a  ceiling  effect  where  as  DANA 
throughput  does  not. 


Human  Inspired.  Technology  Driven. 


32 


Sample  Patient  #1 


CS  Throughput 


MS  Throughput 


In  this  patient,  MMSE  =  30, 
but  DANA  shows  cognitive 
improvement  over  the 
course  of  treatment. 


& 
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Sample  Patient  #2 


PRT_Throughput 


5<H - 1 - 1 - 1 - 1 

0  10  20  30  40 


MS_Throughput 


20- 


oH - 1 - 1 - 1 - 1 

0  10  20  30  40 


CS_Throughput 

80-1 


oH - 1 - 1 - 1 - 1 

0  10  20  30  40 


M  MSE_Score_(_/30) 

In  this  patient,  MMSE  =  30, 

but  DANA  shows  cognitive  - -  -  .. 

decline  over  the  course  of  29- 

treatment. 

27-| - 1 - 1 - 1 - 1 

0  10  20  30  40 
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DANA  Partner  Studies 


& 


US  Coast  Guard:  POC:  Michael  Doria.  Title:  Neurocognitive 
assessment  DoD/NCAA  Sports  Concussion  study 

•  Navy  Experimental  Diving  Unit:  POC:  Jay  Haran.  Title: 
Environmental  testing  to  verify  test-retest  reliability  among  the 
same  subjects  in  different  environments. 

•  National  Intrepid  Center  of  Excellence  (NICoE):  POC:  Douglas 
Brungart.  Title:  Assessing  the  Impact  of  Blast-Related  Sensory 
Impairment  on  Multisensory  Integration  While  Maneuvering  on 
Foot 

•  Walter  Reed  Army  Institute  of  Research  (WRAIR):  POC:  Gary 
Kamimori.  Title:  Impact  of  caffeine  on  neurocognitive 
performance  during  sleep  deprivation  using  the  DANA. 

•  VA-New  Jersey  Health  Care  System  (NJHCS) :  POC:  Rick 
Servatius.  PTSD/TBI  Study. 
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'hank  You! 


clathan@atinc.com 

240-498-9471 
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GO/NO-GO  (GNG) 
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A  figure  will  appear  in  a  window  of 
the  building  below. 


Foe  Friend 


Tap  FIRE  if  the  figure  is  a  foe. 

Do  nothing  if  the  figure  is  a  friend. 


Press  FIRE  to  start. 


i  fe  i 
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Memorize  the  list  of  letters  below. 


STERNBERG  MEMORY  SEARCH 
(STN) 


Z  DTRW 


When  the  section  begins,  single 
letters  from  the  list  will  be  shown. 


Tap  Yes  if  the  Tap  No  if  the 
letter  shown  letter  shown 
was  in  the  list.  was  not  in  the 


list. 

Press  a  button  to  start. 
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SPATIAL  DISCRIMINATION 
(SPD) 
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Match  to  Sample 
(MSP) 


Human  Inspired. 


Psychological  Screening  Instrumeh 

-  ISI  (Insomnia  Screening  Index) 

-  PC-PTSD  (4  item  screener) 


In  the  past  month,  how  much  have 
you  been  bothered  by 

Repeated,  disturbing  memories, 
thoughts,  or  images  of  a  stressful 
military  experience? 

A  Not  at  all 


Psychological  Clinical  Measures: 

-  PCL-m  (PTSD) 

-  PHQ-8/9  (depression) 

-  PSQI  (insomnia). 

-  Deployment  Stress  Inventory  (D 

(NBSI  +  anger/pain/distress) 


little  bit 

^Moderately 
^  Quite  a  bit 
^  Extremely 


Combat  Exposure  Scale  (CES) 


MACE 
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University  of  Wisconsin  -  Brief  Report  (Controls  versus  Concussed) 
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Methods . 2 

Results . 2 

Control  Baseline  versus  Concussed  At  Each  Time  Interval. . 2 

Control  versus  Concussed  At  The  Same  Time  Intervals . 6 

Conclusion . 10 

Figures  and  Tables . 11 


Research  Questions 

1 .  Is  there  a  difference  in  cognitive  performance  between  controls  and  concussed  University 
of  Wisconsin  athletes? 

Participants 

31  athletes  from  the  University  of  Wisconsin  completed  DANA  Standard.  These  31 
athletes  are  comprised  of  two  groups.  Nineteen  of  these  athletes  were  part  of  the  control  who  are 
made  up  of  athletes  who  did  not  experience  a  concussion  during  the  study  period.  The  remaining 
22  athletes  made  up  the  concussed  group.  These  participants  experienced  a  concussion  during 
the  study  period.  Each  participant  completed  DANA  Standard  to  obtain  baseline  measures  in 
cognitive  performance.  Members  of  the  control  group  completed  DANA  Standard,  again,  8,  15, 
and  45  days  after  completing  DANA  Standard  the  first  time  (i.e.  at  baseline).  Members  of  the 
concussed  group  completed  DANA  Standard  within  24-hours  of  experiencing  a  concussion  and 
then  8,  15,  and  45  days  post  concussion. 

Nineteen  athletes  in  the  control  group  completed  DANA  Standard  at  baseline.  Of  these 
19  participants,  18  completed  DANA  Standard  8  days  after  baseline.  Of  the  19  participant  who 
completed  DANA  Standard  at  baseline,  14  completed  DANA  Standard  15  days  after  baseline. 
Finally,  of  the  original  19  participants,  10  completed  DANA  Standard  45  days  after  baseline.  Of 
the  concussed  group,  22  athletes  completed  DANA  Standard  prior  to  experiencing  a  concussion. 
Of  these  22  participants,  8  completed  DANA  Standard  within  24  hours  of  experiencing  a 
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concussion.  Of  the  original  22  participants  who  completed  DANA  Standard  at  baseline,  16 
completed  DANA  Standard  8  days  post  concussion.  Of  the  original  22  participants,  14 
completed  DANA  Standard  15  days  post  concussion.  Finally,  of  the  original  22  participants,  10 
completed  DANA  Standard  45  days  post  concussion. 

Methods 

Anderson-Darling  tests  were  performed  to  assess  the  normality  of  the  data  in  order  to 
determine  which  results,  t-tests  for  normally  distributed  data  and  Wilcoxon  ranksum  for  non- 
normally  distributed  data,  to  use  to  evaluate  whether  or  not  cognitive  performance  differed 
between  the  control  group  and  the  concussed  group.  T-tests  and  Wilcoxon  ranksum  were  run  to 
determine  whether  or  not  significant  differences  in  cognitive  performance  exist  between  control 
participants  and  concussed  participants.  First,  analyses  were  run  comparing  control  participants’ 
baseline  cognitive  performance  to  concussed  participants’  cognitive  performance  at  each  time 
interval  (i.e.  24  hours,  8  days,  15  days,  and  45  days  post  concussion).  Secondly,  analyses  were 
run  comparing  non-concussed  participants’  and  concussed  participants’  cognitive  performance  at 
the  same  time  intervals  (i.e.  baseline,  8  days,  15  days,  and  45  days). 

Results 

Control  Baseline  versus  Concussed  At  Each  Time  Interval 

Mean  Reaction  Time.  The  Anderson-Darling  tests  revealed  that  the  mean  reaction  times 
for  the  control  group  baselines  for  all  DANA  subtests  with  the  exception  of  the  first 
administration  of  Simple  Reaction  Time  (RT1)  are  normally  distributed.  Furthermore,  these 
same  tests  revealed  that  mean  reaction  times  for  the  concussed  group  are  normally  distributed 
with  the  exception  of  the  following:  RT1  at  24  hours  and  15  days  post  concussion,  Code 
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Substitution  Delayed  (CSD)  8  days  post  concussion,  Procedural  Reaction  Time  (PRT)  15  days 
post  concussion,  and  Memory  Search  (MS)  15  and  45  days  post  concussion. 

The  t-tests  and  Wilcoxon  ranksum  results  for  mean  reaction  times  are  the  same  regardless 
of  the  normality  of  the  data.  With  this  in  mind,  the  results  in  Table  1  indicate  that  there  is  no 
difference  in  mean  reaction  time  for  any  DANA  subtests  between  the  control  group  baselines 
and  the  concussed  group  24  hours  post  concussion  (ps  >  .05).  The  results  are  similar  for  mean 
reaction  times  between  the  control  group  baselines  and  the  concussed  group  8  days  post 
concussion  (ps  >  .05).  Table  2  shows  no  significant  differences  in  mean  reaction  time  between 
these  two  groups  for  any  DANA  subtest  (ps  >  .05).  The  comparison  between  the  control  group 
baselines  and  the  concussed  group  15  days  post  concussion  (Table  3)  show  that  these  two  groups 
exhibited  a  near  significant  difference  in  mean  reaction  times  on  CSD  (Mcontroi  =  1044.58, 
SDcontroi  =  185.99,  Mconcussed  =  1 192.38,  SDC oncussed  =  255.47)  (t  =  -1.93;  Pt-test  <  .10).  However, 
no  other  significant  differences  between  mean  reaction  times  for  the  aforementioned  two  groups 
exist  based  on  the  results  shown  in  Table  3  (ps  >  .05).  Finally,  Table  4  demonstrates  that  no 
significant  differences  in  mean  reaction  times  exist  between  the  control  group  baselines  and  the 
concussed  group  45  days  post  concussion  for  any  DANA  subtest  (ps  >  .05). 

Mean  Throughput.  The  Anderson-Darling  tests  show  that  mean  throughput  for  the 
control  group  baselines  for  all  DANA  subtests  with  the  exception  of  Match  To  Sample  (MTS) 
are  normally  distributed.  Additionally,  these  same  tests  show  that  mean  throughput  for  the 
concussed  group  are  normally  distributed  with  the  exception  of  the  following:  RT1  24  hours 
post  concussion,  MS  1 5  days  post  concussion,  and  MTS  45  days  post  concussion. 

The  t-tests  and  Wilcoxon  ranksum  results  for  mean  throughput  are  similar  regardless  of 
normality.  The  results  in  Table  5  indicate  that  there  is  no  difference  in  mean  throughput  for  any 
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DANA  subtests  between  the  control  group  baselines  and  the  concussed  group  24  hours  post 
concussion  (ps  >  .05).  The  results  are  similar  for  mean  throughput  between  the  control  group 
baselines  and  the  concussed  group  8  days  post  concussion.  Table  6  shows  no  significant 
differences  in  mean  throughput  between  these  two  groups  for  any  DANA  subtest  (ps  >  .05).  The 
comparison  between  the  control  group  baselines  and  the  concussed  group  15  days  post 
concussion  (Table  7)  show  that  these  two  groups  differ  significantly  in  mean  throughput  on  CSD 
(A/control  =  55.88,  iSDcontrol  =  9.55,  Mconcussed  =  48.37,  SDconcussed  =  9.56)  (t  =  9.53;  pt-test  <  .05)\ 
Thus,  indicating  that  non-concussed  participants  have  greater  mean  throughput  on  CSD  than 
concussed  participants  who  complete  CSD  15  days  following  concussion.  However,  no  other 
significant  differences  between  mean  throughput  for  the  aforementioned  two  groups  exist  based 
on  the  results  shown  in  Table  7  (ps  >  .05).  Finally,  Table  8  demonstrates  that  no  significant 
differences  in  mean  throughput  exist  between  the  control  group  baselines  and  the  concussed 
group  45  days  post  concussion  for  any  DANA  subtest  (ps  >  .05). 

Percent  Correct.  Unlike  mean  reaction  times  and  mean  throughput,  Anderson-Darling 
tests  demonstrate  that  all  of  the  percent  correct  data  for  the  control  group  baselines  is  not 
normally  distributed  for  all  DANA  subtests.  Furthermore,  Anderson-Darling  tests  demonstrate 
that  the  majority  of  the  percent  correct  data  for  the  concussed  group  is,  also,  not  normally 
distributed  for  most  of  the  DANA  subtests  with  the  exception  of  the  following:  Code 
Substitution  Learning  (CSL)  24  hours,  8  and  45  days  post  concussion,  PRT  24  hours  post 
concussion,  MTS  24  hours,  8  and  45  days  post  concussion,  MS  24  hours  and  1 5  days  post 
concussion,  and  Spatial  Processing  (SP)  1 5  days  post  concussion. 
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T-test  results  rather  than  Wilcoxon  ranksum  are  provided  because  data  from  both  groups  is  normally  distributed  as 
reported  in  the  previous  paragraph. 


5 


The  findings  for  percent  correct  are  similar  to  those  for  mean  throughput,  for  the  most 
part  (see  Tables  9  through  12),  regardless  of  data  normality.  Table  9  indicates  that  there  is  a  near 
significant  difference  in  GNG  percent  correct  between  the  control  group  baselines  (M=  97.72, 
SD  =  2.73)  and  the  concussed  group  24  hours  post  concussion  (M=  99.58,  SD  =  1.18) 

(pwiicoxon  <  .10)2.  The  results  also  reveal  a  significant  difference  in  CSD  percent  correct  between 
the  control  group  baselines  (M=  92.84,  SD  =  6.16)  and  the  concussed  group  24  hours  post 
concussion  (M=  89.24,  SD  =  3.13)  (pwncoxon  <  .05).  Consequently,  this  finding  suggests  that  24 
hours  post  concussion  participants  respond  correctly  to  fewer  questions  on  CSD  than  non- 
concussed  participants.  Table  9  reveals  that  there  are  no  other  significant  differences  in  percent 
correct  between  the  control  group  baselines  and  the  concussed  group  24  hours  post  concussion 
on  the  remaining  DANA  subtests  (ps  >  .  10). 

The  second  administration  of  Simple  Reaction  Time  (RT2)  percent  correct  is 
significantly  different  between  the  control  group  baselines  (M=  97.24,  SD  =  2.99)  and  the 
concussed  group  8  days  post  concussion  (M=  99.22,  SD  =  1.51)  (pwncoxon  <  .05),  as  shown  in 
Table  10.  Surprisingly,  this  result  indicates  that  the  concussed  group  responds  correctly  more 
often  on  RT2  than  non-concussed  participants.  Again,  no  significant  differences  in  percent 
correct  between  the  control  group  baselines  and  the  concussed  group  8  days  post  concussion  is 
illustrated  in  Table  10  (ps  >  .10). 

The  results  regarding  percent  correct  between  the  control  group  baselines  and  the 
concussed  group  15  days  post  concussion  are  shown  in  Table  11.  The  outcomes  shown  in  this 
table  demonstrate  that  RT1  percent  correct  differs  significantly  between  the  control  group 
baselines  (M=  99.61,  SD  =  1.25)  and  the  concussed  group  15  days  post  concussion  (M=  96.79, 

2  Wilcoxon  ranksum  results  are  provided  because  data  from  both  groups  is  not  normally  distributed  as  reported  in 
the  previous  paragraph. 
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SD  =  5.41)  (pwiicoxon  <  .05).  This  outcome  implies  that  15  days  following  concussion,  concussed 
participants  respond  more  incorrectly  on  RT1  than  non-concussed  participants.  Table  1 1  also 
reveals  a  nearly  significant  difference  on  PRT  percent  correct  between  the  control  group 
baselines  (M=  95.56,  SD  =  3.35)  and  the  concussed  group  15  days  following  concussion  (M  = 
97.55,  SD  =  2.79)  (pwiicoxon  <  .10).  No  significant  differences  are  demonstrated  on  percent 
correct  between  the  control  group  baselines  and  the  concussed  group  15  days  post  concussion  on 
any  of  the  remaining  DANA  subtests  (ps  >  .10). 

Finally,  Table  12  shows  a  significant  difference  in  CSL  percent  correct,  PRT  percent 
correct,  and  GNG  percent  correct  between  the  control  group  baselines  (M=  94.44,  SD  =  3.76;  M 
=  95.56,  SD  =  3.35;  M=  97.72,  SD  =  2.73;  respectively)  and  the  concussed  group  45  days  post 
concussion  (M=  98.06,  SD  =  1.63;  M=  98.13,  SD  =  2.19;  M=  99.67,  SD  =  1.05;  respectively) 
(pswiicoxon  <  .05).  Each  of  these  results  suggest  that  45  days  following  concussion,  concussed 
participants  respond  more  correctly  on  CSL,  PRT,  and  GNG  than  non-concussed  participants. 
None  of  the  percent  correct  scores  on  the  remaining  DANA  subtests  are  significantly  different 
between  the  control  group  baselines  and  the  concussed  group  45  days  post  concussion. 

Control  versus  Concussed  At  The  Same  Time  Intervals 

Mean  Reaction  Time.  The  Anderson-Darling  tests  revealed  that  the  mean  reaction  times 
for  the  control  group  are  normally  distributed  for  all  DANA  subtests  with  the  exception  of  the 
following:  RT1  at  baseline  and  8  and  15  days  post  baseline,  and  PRT  8  and  15  days  post 
baseline,  and  SP  8  days  post  baseline.  Furthermore,  the  Anderson-Darling  tests  revealed  that 
mean  reaction  times  for  the  concussed  group  are  normally  distributed  with  the  exception  of  the 
following:  GNG  at  baseline,  CSD  8  days  post  concussion,  RT1  15  days  post  concussion,  PRT 
15  days  post  concussion,  and  MS  15  and  45  days  post  concussion. 
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The  results  in  Table  13  indicate  that  there  is  a  significant  difference  in  baseline  CSD 
mean  reaction  time  between  the  control  group  (M  =  1044.58,  SD  =  185.99)  and  the  concussed 
group  (M  =  1 189.62,  SD  =  225.44)  (t  =  -2.23,  p  <  .05)  at  baseline.  This  outcome  suggests  that 
CSD  mean  reaction  time  is  slower,  on  average,  for  the  concussed  group  before  experiencing  a 
concussion  than  for  the  control  group.  Table  13  shows  that  there  are  no  significant  differences  in 
baseline  mean  reaction  times  between  the  control  group  and  the  concussed  group  on  any  of  the 
remaining  DANA  subtests  (ps  >  .05).  There  is  no  difference  in  mean  reaction  times  between  the 
control  group  8  days  post  baseline  and  the  concussed  group  8  days  post  concussion  for  any 
DANA  subtests  (ps  >  .05),  as  shown  in  Table  14. 

On  the  other  hand,  Table  15  illustrates  that  there  is  a  near  significant  difference  in  CSL 
and  SP  mean  reaction  time  between  the  control  group  15  days  post  baseline  (M  =  1099.02, 

SD  =  144.13;  M=  1226.49,  SD  =  208.42;  respectively)  and  the  concussed  group  15  days  post 
concussion  (M  =  1366.80,  SD  =  248.20;  M=  1540.89,  SD  =  276.29;  respectively)  ( t  =  -1.88, 
p  <  .10;  t  =  -\.75,p  <  .10;  respectively).  Conversely,  Table  15  also  shows  that  there  is  no 
significant  difference  in  mean  reaction  times  15  days  post  baseline  for  the  control  group  and  15 
days  post  concussion  for  the  concussed  group  on  any  of  the  remaining  DANA  subtests 
(ps  >  .05).  Finally,  mean  reaction  times  between  the  control  group  45  days  post  baseline  and  the 
concussed  group  45  days  post  concussion  are  not  significantly  different  (ps  >  .05),  as  shown  in 
Table  16. 

Mean  Throughput.  The  Anderson-Darling  tests  show  that  mean  throughput  for  the 
control  group  for  all  DANA  subtests  are  normally  distributed  with  the  exception  of  the 
following:  MTS  at  baseline,  RT1  8  days  post  baseline,  and  PRT  8  days  post  baseline. 
Additionally,  the  Anderson-Darling  tests  show  that  mean  throughput  for  the  concussed  group  are 
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normally  distributed  with  the  exception  of  the  following:  GNG  at  baseline,  CSD  at  baseline,  MS 
15  days  post  concussion,  and  MTS  45  days  post  concussion.  The  only  significant  difference  in 
mean  throughput  between  the  control  group  and  the  concussed  group  occurs  at  baseline  on  CSD 
(M=  55.88,  SD  =  9.55;  M=  46.94,  SD  =  9.13;  respectively)  (pwiicoxon  <  .05),  shown  in  Table  17. 
This  finding  implies  that  CSD  mean  throughput  is  greater  for  the  control  group  than  for  the 
concussed  group  prior  to  experiencing  a  concussion.  Table  17  also  shows  that  there  are  no 
significant  differences  in  mean  throughput  for  any  remaining  DANA  subtest  at  baseline 
(ps  >  .05).  Moreover,  there  are  no  significant  differences  in  mean  throughput  on  any  DANA 
subtests  between  the  control  group  at  8,  15,  and  45  days  post  baseline  and  the  concussed  group  8, 
15,  and  45  days  post  concussion  (ps  >  .05),  respectively,  as  shown  in  Tables  18  through  20. 

Percent  Correct.  Unlike  mean  reaction  times  and  mean  throughput,  Anderson-Darling 
tests  demonstrate  that  all  of  the  percent  correct  data  for  the  control  group  is  not  normally 
distributed  for  all  DANA  subtests  with  the  exception  of  the  following:  CSL  8  and  45  days  post 
baseline,  SP  8  and  15  days  post  baseline,  CSD  15  days  post  baseline,  and  MTS  45  days  post 
baseline.  Furthermore,  Anderson-Darling  tests  demonstrate  that  the  majority  of  the  percent 
correct  data  for  the  concussed  group  is,  also,  not  normally  distributed  for  the  DANA  subtests 
with  the  exception  of  the  following:  CSD  at  baseline  and  15  days  post  concussion,  CSL  8  and  45 
days  post  concussion,  MTS  8,  15,  and  45  days  post  concussion,  and  SP  15  days  post  concussion. 

The  results  of  percent  correct  between  control  and  concussed  groups  at  equivalent  time 
intervals  are  provided  in  Tables  21  through  24.  Beginning  with  baseline  percent  correct,  Table 
21  shows  that  baseline  RT2,  CSD,  and  MS  percent  correct  scores  are  significantly  different 
between  the  control  group  (M=  97.24,  SD  =  2.99;  M=  92.84,  SD  =  6.16;  M=  82.63,  SD  =  19.89; 
respectively)  and  the  concussed  group  (M  =  98.64,  SD  =  3.06;  M=  88.76,  SD  =  7.30;  M  =  71.21, 


9 


SD  =  23.27;  respectively)  (pwiicoxonS  <  .05).  These  results  suggests  that  percent  correct  on  RT2  is 
higher  for  the  concussed  group  before  sustaining  a  concussion  than  the  control  group  at  baseline. 
Regarding  CSD  and  MS,  the  aforementioned  significant  results  imply  that  percent  correct  on 
CSD  and  MS  is  lower  for  the  concussed  group  before  sustaining  a  concussion  than  for  the 
control  group  at  baseline.  Additionally,  Table  21  shows  that  CSL  percent  correct  scores  are 
nearly  significantly  different  between  the  control  group  and  the  concussed  group  at  baseline 
(pwiicoxon  <  .10).  Lastly,  Table  21  indicates  that  there  are  no  significant  differences  in  percent 
correct  between  the  control  group  and  the  concussed  group  at  baseline  on  any  of  the  remaining 
DANA  subtests  (pwiicoxonS  >  .05). 

Percent  correct  scores  8  days  post  baseline  for  the  control  group  and  8  days  post 
concussion  for  the  concussed  group  are  not  significantly  different  (pwncoxonS  >  .05),  as  shown  in 
Table  22.  The  results  are  similar  for  percent  correct  scores  15  days  post  baseline  and  15  days 
post  concussion,  as  shown  in  Table  23  (pt-testS  >  .05  for  SP  and  CSD;  /iwiicoxonS  >  .05  for 
remaining  DANA  subtests)3.  Table  24  presents  the  results  of  percent  correct  45  days  post 
baseline  for  the  control  group  and  45  days  post  concussion  for  the  concussed  group.  As  shown, 
these  two  groups  differ  significantly  on  percent  correct  scores  on  RT1  (M=  98.65,  SD  =  2.19; 
M=  100.00,  SD  =  0.00;  respectively)  (pwiicoxon  <  .05).  This  outcome  suggests  that  percent 
correct  is  higher  for  concussed  participants  45  days  post  concussion  than  for  non-concussed 
participants  45  days  post  baseline.  Along  with  these  findings,  Table  24  demonstrates  that 
percent  correct  scores  are  nearly  significantly  different  between  the  control  group  45  days  post 
baseline  and  the  concussed  group  45  days  post  concussion  on  CSL  (M=  95.51,  SD  =  4.21; 

M=  98.06,  SD  =  1.63;  respectively)  and  SP  (M=  89.62,  SD  =  7.21;  M=  94.50,  SD  =  4.38; 

3  Percent  correct  for  SP  and  CSD  for  both  groups  is  normally  distributed,  as  reported  earlier,  thus  t-test  results  are 
reported. 
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respectively)  (pt-tests  <  .10).  Finally,  Table  24  shows  no  significant  differences  in  percent  correct 
scores  between  the  control  group  45  days  post  baseline  and  the  concussed  group  45  days  post 
concussion  on  any  of  the  remaining  DANA  subtests  (ps  >  .05). 

Conclusion 

Research  Question  1.  There  is  a  significant  difference  in  cognitive  performance  between 
the  control  group  baselines  and  the  concussed  group  at  various  time  intervals  on  certain  aspects 
of  DANA  subtests.  Control  group  baselines  on  CSD  mean  throughput  is  higher,  on  average, 
than  CSD  mean  throughput  for  the  concussed  group  15  days  post  concussion.  Percent  correct  at 
baseline  for  the  control  group,  on  average,  is  higher  than  the  concussed  group  CSD  percent 
correct  24  hours  post  concussion.  Interestingly,  RT2  percent  correct  at  baseline  for  the  control 
group  is  lower,  on  average  than  percent  correct  for  the  concussed  group  8  days  post  concussion. 

Much  like  with  the  control  group  baselines  versus  the  concussed  group  at  different  time 
intervals,  control  group  cognitive  performance  at  equivalent  time  points  as  that  of  the  concussed 
group  are  significantly  different  for  certain  aspects  of  DANA  subtests.  Mean  reaction  time  at 
baseline  for  the  control  group  is  faster,  on  average,  than  for  the  concussed  group  at  baseline. 
Furthermore,  mean  throughput  at  baseline  for  the  control  group  is  higher,  on  average,  than  for 
the  concussed  group  at  baseline.  Finally,  RT2  percent  correct  is  lower,  on  average,  for  the 
control  group  at  baseline  than  for  the  control  group  at  baseline.  On  the  other  hand,  CSD  and  MS 
percent  correct  at  baseline  is  higher,  on  average,  for  the  control  group  at  baseline  than  for  the 
control  group  at  baseline.  Additionally,  RT1  percent  correct  45  days  post  baseline  for  the 
control  group  is  lower,  on  average,  than  for  the  concussed  group  45  days  post  concussion. 
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Figures  and  Tables 


Control  Baseline  vs.  24  Hours  Post  Concussion 


Table  1.  Control  Baselines  vs.  Concussed  (24-hour  Post  Concussion):  Mean  Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  (M) 

meanRT  ( SD ) 

meanRT  (M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

331.591 

60.778 

342.003 

69.620 

63.378 

-0.390 

0.700 

0.577 

RT2 

352.372 

64.683 

373.685 

58.672 

63.058 

-0.802 

0.430 

0.541 

CSL 

1191.656 

209.185 

1275.963 

91.619 

184.001 

-1.087 

0.287 

0.105 

PRT 

609.510 

96.054 

659.781 

66.888 

88.858 

-1.342 

0.192 

0.130 

SP 

1637.368 

349.335 

1766.275 

301.114 

336.530 

-0.909 

0.372 

0.353 

GNG 

559.446 

88.338 

585.635 

80.464 

86.206 

-0.721 

0.478 

0.473 

CSD 

1044.580 

185.994 

1172.714 

256.469 

208.147 

-1.461 

0.157 

0.159 

MTS 

1599.602 

368.970 

1770.663 

165.192 

325.055 

-1.249 

0.223 

0.212 

MS 

880.956 

158.446 

804.101 

142.501 

154.148 

1.183 

0.248 

0.614 
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Control  Baseline  vs.  8  Days  Post  Concussion 
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1400 
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§ 
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RT1 

RT2 

CSL 

PRT 

SP 

GNG 

CSD 

MTS 

MS 

*  control  average  meanRT 

331.591 

352.372 

1191.656 

609.510 

1637.368 

559.446 

1044.580 

1599.602 

880.956 

■  concussed  average  meanRT 

338.555 

336.663 

1220.429 

618.361 

1611.781 

559.424 

1185.015 

1685.681 

854.047 

Table  2.  Control  Baselines  vs.  Concussed  (8  Days  Post  Concussion):  Mean  Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  (M) 

meanRT  ( SD ) 

meanRT  (M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

331.591 

60.778 

338.555 

52.533 

57.178 

-0.359 

0.722 

0.540 

RT2 

352.372 

64.683 

336.663 

44.591 

56.444 

0.820 

0.418 

0.417 

CSL 

1191.656 

209.185 

1220.429 

223.772 

215.938 

-0.393 

0.697 

0.855 

PRT 

609.510 

96.054 

618.361 

90.298 

93.482 

-0.279 

0.782 

0.703 

SP 

1637.368 

349.335 

1611.781 

354.023 

351.474 

0.215 

0.831 

0.655 

GNG 

559.446 

88.338 

559.424 

90.291 

89.231 

0.001 

0.999 

0.830 

CSD 

1044.580 

185.994 

1185.015 

344.811 

270.023 

-1.533 

0.135 

0.362 

MTS 

1599.602 

368.970 

1685.681 

329.130 

351.421 

-0.722 

0.475 

0.631 

MS 

880.956 

158.446 

854.047 

259.578 

210.526 

0.377 

0.709 

0.562 

■^Control  ^Concussed  16. 
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Control  Baseline  vs.  15  Days  Post  Concussion 
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«  2 

a  e- 

CG 

a> 


2000 


1500 


1000 


500 


u 

RT1 

RT2 

CSL 

PRT 

SP 

GNG 

CSDf 

MTS 

MS 

■*  control  average  meanRT 

331.591 

352.372 

1191.656 

609.510 

1637.368 

559.446 

1044.580 

1599.602 

880.956 

■  concussed  average  meanRT 

344.734 

344.083 

1226.491 

637.196 

1540.886 

578.141 

1192.381 

1771.264 

799.697 

Table  3.  Control  Baselines  vs.  Concussed  (15  Days  Post  Concussion):  Mean  Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  (M) 

meanRT  ( SD ) 

meanRT  (M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

331.591 

60.778 

344.734 

71.621 

65.544 

-0.569 

0.573 

0.623 

RT2 

352.372 

64.683 

344.083 

47.847 

58.218 

0.404 

0.689 

0.785 

CSL 

1191.656 

209.185 

1226.491 

208.422 

208.865 

-0.474 

0.639 

0.372 

PRT 

609.510 

96.054 

637.196 

154.840 

124.143 

-0.633 

0.531 

0.870 

SP 

1637.368 

349.335 

1540.886 

276.292 

320.736 

0.854 

0.400 

0.334 

GNG 

559.446 

88.338 

578.141 

111.317 

98.628 

-0.538 

0.594 

0.785 

CSD 

1044.580 

185.994 

1192.381 

255.468 

217.843 

-1.926 

0.063 

0.071 

MTS 

1599.602 

368.970 

1771.264 

408.165 

385.892 

-1.263 

0.216 

0.353 

MS 

880.956 

158.446 

799.697 

218.689 

186.099 

1.240 

0.224 

0.572 

^Control  19,  ^Concussed  14. 
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Control  Baseline  vs.  45  Days  Post  Concussion 


Table  4.  Control  Baselines  vs.  Concussed  (45  Days  Post  Concussion):  Mean  Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  ( M) 

meanRT  ( SD ) 

meanRT  ( M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

331.591 

60.778 

333.873 

36.570 

53.930 

-0.108 

0.915 

0.551 

RT2 

352.372 

64.683 

349.610 

38.211 

57.236 

0.124 

0.903 

1.000 

CSL 

1191.656 

209.185 

1223.332 

169.054 

196.720 

-0.412 

0.683 

0.536 

PRT 

609.510 

96.054 

627.068 

106.451 

99.640 

-0.451 

0.656 

0.766 

SP 

1637.368 

349.335 

1521.850 

239.655 

317.020 

0.933 

0.359 

0.281 

GNG 

559.446 

88.338 

583.790 

95.360 

90.739 

-0.687 

0.498 

0.663 

CSD 

1044.580 

185.994 

1168.890 

293.503 

227.546 

-1.398 

0.173 

0.281 

MTS 

1599.602 

368.970 

1725.830 

269.019 

338.944 

-0.953 

0.349 

0.371 

MS 

880.956 

158.446 

886.296 

153.209 

156.720 

-0.087 

0.931 

0.396 

^Control  19,  ^Concussed  10- 
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Table  5.  Control  Baselines  vs.  Concussed  (24-hours  Post  Concussion):  Mean  Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThru 

(M) 

meanThru 

C SD ) 

meanThru 

M 

meanThru 

(SD) 

f  (SD) 

t 

P 

P 

RT1 

185.608 

29.594 

179.684 

27.776 

29.096 

0.483 

0.633 

0.577 

RT2 

172.687 

30.439 

162.979 

26.526 

29.396 

0.784 

0.441 

0.690 

CSL 

48.948 

7.085 

46.054 

2.826 

6.195 

1.108 

0.278 

0.212 

PRT 

96.287 

13.929 

89.329 

8.712 

12.686 

1.301 

0.205 

0.212 

SP 

35.093 

7.200 

32.157 

6.454 

6.999 

0.996 

0.329 

0.326 

GNG 

106.746 

15.947 

103.920 

15.809 

15.908 

0.421 

0.677 

0.730 

CSD 

55.879 

9.550 

49.306 

8.277 

9.211 

1.693 

0.103 

0.105 

MTS 

36.231 

11.311 

32.295 

3.664 

9.792 

0.954 

0.349 

0.730 

MS 

60.312 

17.224 

52.548 

24.415 

19.507 

0.944 

0.354 

0.507 

^Control  — 

19,  ^Concussed  —  8. 
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Control  Baseline  vs.  8  Days  Post  Concussion 


Table  6.  Control  Baselines  vs.  Concussed  (8  Days  Post  Concussion):  Mean  Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThru 

{M) 

meanThru 

(SD) 

meanThru 

(M) 

meanThru 

(SD) 

t  (SD) 

t 

P 

P 

RT1 

185.608 

29.594 

180.851 

26.936 

28.416 

0.493 

0.625 

0.540 

RT2 

172.687 

30.439 

180.301 

22.940 

27.287 

-0.822 

0.417 

0.345 

CSL 

48.948 

7.085 

49.114 

9.357 

8.196 

-0.060 

0.953 

0.855 

PRT 

96.287 

13.929 

94.307 

12.687 

13.379 

0.436 

0.666 

0.728 

SP 

35.093 

7.200 

34.729 

5.938 

6.656 

0.161 

0.873 

0.804 

GNG 

106.746 

15.947 

108.107 

17.424 

16.635 

-0.241 

0.811 

0.987 

CSD 

55.879 

9.550 

50.396 

13.652 

11.596 

1.394 

0.173 

0.329 

MTS 

36.231 

11.311 

35.154 

7.657 

9.820 

0.323 

0.748 

0.804 

MS 

60.312 

17.224 

58.429 

19.936 

18.506 

0.300 

0.766 

0.655 

^Control  19)  -A^Concussed  16- 
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Control  Baseline  vs.  15  Days  Post  Concussion 


Table  7.  Control  Baselines  vs.  Concussed  (15  Days  Post  Concussion):  Mean  Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThra 

(M) 

meanThru 

C SD ) 

meanThru 

(M) 

meanThru 

(SD) 

t  (SD) 

t 

P 

P 

RT1 

185.608 

29.594 

177.161 

29.404 

29.514 

0.813 

0.423 

0.455 

RT2 

172.687 

30.439 

176.109 

24.952 

28.268 

-0.344 

0.733 

0.675 

CSL 

48.948 

7.085 

48.607 

9.416 

8.144 

0.119 

0.906 

0.548 

PRT 

96.287 

13.929 

95.649 

16.743 

15.173 

0.119 

0.906 

0.757 

SP 

35.093 

7.200 

37.267 

6.087 

6.755 

-0.913 

0.368 

0.299 

GNG 

106.746 

15.947 

103.678 

20.853 

18.166 

0.479 

0.635 

0.913 

CSD 

55.879 

9.550 

48.372 

9.558 

9.553 

2.231 

0.033 

0.051 

MTS 

36.231 

11.311 

32.250 

8.158 

10.109 

1.118 

0.272 

0.392 

MS 

60.312 

17.224 

63.617 

22.985 

19.845 

-0.473 

0.640 

0.597 

^Control  19,  ^Concussed  14. 
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Control  Baseline  vs.  45  Days  Post  Concussion 


Table  8.  Control  Baselines  vs.  Concussed  (45  Days  Post  Concussion):  Mean  Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThra 

C M) 

meanThru 

C SD ) 

meanThru 

C M) 

meanThru 

(SD) 

t  (SD) 

t 

P 

P 

RT1 

185.608 

29.594 

181.594 

19.232 

26.592 

0.386 

0.702 

0.614 

RT2 

172.687 

30.439 

170.558 

20.539 

27.538 

0.198 

0.845 

0.836 

CSL 

48.948 

7.085 

48.908 

7.117 

7.096 

0.015 

0.988 

0.801 

PRT 

96.287 

13.929 

96.188 

13.584 

13.815 

0.018 

0.986 

0.836 

SP 

35.093 

7.200 

38.343 

6.220 

6.889 

-1.207 

0.238 

0.113 

GNG 

106.746 

15.947 

105.007 

17.569 

16.505 

0.270 

0.789 

0.801 

CSD 

55.879 

9.550 

49.597 

13.770 

11.135 

1.444 

0.160 

0.302 

MTS 

36.231 

11.311 

33.773 

6.763 

10.027 

0.627 

0.536 

0.909 

MS 

60.312 

17.224 

62.491 

19.421 

17.986 

-0.310 

0.759 

0.836 

^Control  19,  ^Concussed  10. 


19 


Control  Baseline  vs.  24  Hours  Post  Concussion 


100 
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Q  O 
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/u 

RT 

RT 

CSLt 

PRT 

SP 

GNG| 

CSD* 

MTS 

MS 

»  control  average  pctCor 

99.605 

97.237 

94.444 

95.559 

91.842 

97.719 

92.836 

87.368 

82.632 

11  concussed  average  pctCor 

98.438 

95.625 

96.875 

96.875 

91.875 

99.583 

89.236 

89.167 

72.083 

Table  9.  Control  Baselines  vs.  Concussed  (24-hour  Post  Concussion):  Percent  Correct. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

pctCor  ( M) 

pctCor  {SD) 

pctCor  ( M) 

pctCor  {SD) 

t  {SD) 

t 

P 

P 

RT1 

99.605 

1.254 

98.438 

2.652 

1.761 

1.574 

0.128 

0.117 

RT2 

97.237 

2.992 

95.625 

8.101 

4.982 

0.768 

0.450 

0.650 

CSL 

94.444 

3.761 

96.875 

1.929 

3.351 

-1.721 

0.098 

0.132 

PRT 

95.559 

3.346 

96.875 

2.362 

3.102 

-1.006 

0.324 

0.410 

SP 

91.842 

7.305 

91.875 

3.720 

6.504 

-0.012 

0.991 

0.742 

GNG 

97.719 

2.734 

99.583 

1.178 

2.402 

-1.841 

0.077 

0.083 

CSD 

92.836 

6.164 

89.236 

3.128 

5.486 

1.557 

0.132 

0.028 

MTS 

87.368 

9.912 

89.167 

6.607 

9.108 

-0.468 

0.644 

0.747 

MS 

82.632 

19.894 

72.083 

34.638 

24.918 

1.004 

0.325 

0.979 

^Control  19,  ^Concussed 
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Control  Baseline  vs.  8  Days  Post  Concussion 


100 


i 


m 

RT1 

RT2* 

CSL 

PRT 

SP 

GNG 

CSD 

MTS 

MS 

control  average  pctCor 

99.605 

97.237 

94.444 

95.559 

91.842 

97.719 

92.836 

87.368 

82.632 

■  concussed  average  pctCor 

99.219 

99.219 

96.094 

95.508 

90.000 

98.750 

90.278 

88.750 

79.375 

Table  10.  Control  Baselines  vs.  Concussed  (8  Days  Post  Concussion):  Percent  Correct. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

pctCor  ( M) 

pctCor  (, SD ) 

pctCor  ( M) 

pctCor  (SD) 

t  (SD) 

t 

P 

P 

RT1 

99.605 

1.254 

99.219 

1.760 

1.505 

0.757 

0.455 

0.496 

RT2 

97.237 

2.992 

99.219 

1.505 

2.432 

-2.402 

0.022 

0.029 

CSL 

94.444 

3.761 

96.094 

3.295 

3.557 

-1.367 

0.181 

0.165 

PRT 

95.559 

3.346 

95.508 

3.948 

3.632 

0.042 

0.967 

0.917 

SP 

91.842 

7.305 

90.000 

7.303 

7.304 

0.743 

0.463 

0.423 

GNG 

97.719 

2.734 

98.750 

1.667 

2.311 

-1.315 

0.198 

0.321 

CSD 

92.836 

6.164 

90.278 

9.185 

7.686 

0.981 

0.334 

0.601 

MTS 

87.368 

9.912 

88.750 

5.821 

8.306 

-0.490 

0.627 

0.933 

MS 

82.632 

19.894 

79.375 

21.815 

20.789 

0.462 

0.647 

0.278 

■^Control  19)  -A^Concussed  16- 
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Control  Baseline  vs.  15  Days  Post  Concussion 
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RT1* 

RT2 

CSL 

PRT| 

SP 

GNG 

CSDf 

MTS 

MS 

m  control  average  pctCor 

99.605 

97.237 

94.444 

95.559 

91.842 

97.719 

92.836 

87.368 

82.632 

■  concussed  average  pctCor 

96.786 

98.393 

96.032 

97.545 

92.143 

96.905 

88.690 

86.905 

84.048 

Table  11.  Control  Baselines  vs.  Concussed  (15  Days  Post  Concussion):  Percent  Correct. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

pctCor  ( M) 

pctCor  {SD) 

pctCor  ( M) 

pctCor  {SD) 

t  {SD) 

t 

P 

P 

RT1 

99.605 

1.254 

96.786 

5.409 

3.631 

2.205 

0.035 

0.013 

RT2 

97.237 

2.992 

98.393 

1.862 

2.580 

-1.272 

0.213 

0.321 

CSL 

94.444 

3.761 

96.032 

3.307 

3.578 

-1.260 

0.217 

0.209 

PRT 

95.559 

3.346 

97.545 

2.789 

3.124 

-1.804 

0.081 

0.065 

SP 

91.842 

7.305 

92.143 

5.447 

6.590 

-0.130 

0.898 

0.895 

GNG 

97.719 

2.734 

96.905 

6.976 

4.974 

0.465 

0.645 

0.656 

CSD 

92.836 

6.164 

88.690 

7.585 

6.796 

1.732 

0.093 

0.105 

MTS 

87.368 

9.912 

86.905 

6.467 

8.636 

0.152 

0.880 

0.671 

MS 

82.632 

19.894 

84.048 

25.726 

22.525 

-0.178 

0.859 

0.658 

■^Control  19,  ^Concussed  14. 
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Control  Baseline  vs.  45  Days  Post  Concussion 


iiirviiii 


/u 

RT 

RT 

CSL* 

PRT* 

SP 

GNG* 

CSD 

MTS 

MS 

control  average  pctCor 

99.605 

97.237 

94.444 

95.559 

91.842 

97.719 

92.836 

87.368 

82.632 

m  concussed  average  pctCor 

100.000 

98.500 

98.055 

98.125 

94.500 

99.667 

88.055 

91.000 

85.000 

Table  12.  Control  Baselines  vs.  Concussed  (45  Days  Post  Concussion):  Percent  Correct. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

pctCor  ( M) 

pctCor  (SD) 

pctCor  ( M) 

pctCor  (SD) 

t  (SD) 

t 

P 

P 

RT1 

99.605 

1.254 

100.000 

0.000 

1.024 

-0.987 

0.332 

0.321 

RT2 

97.237 

2.992 

98.500 

1.748 

2.644 

-1.223 

0.232 

0.328 

CSL 

94.444 

3.761 

98.055 

1.630 

3.212 

-2.878 

0.008 

0.005 

PRT 

95.559 

3.346 

98.125 

2.185 

3.009 

-2.183 

0.038 

0.036 

SP 

91.842 

7.305 

94.500 

4.378 

6.478 

-1.050 

0.303 

0.471 

GNG 

97.719 

2.734 

99.667 

1.054 

2.313 

-2.155 

0.040 

0.044 

CSD 

92.836 

6.164 

88.055 

13.737 

9.393 

1.303 

0.204 

0.483 

MTS 

87.368 

9.912 

91.000 

5.676 

8.731 

-1.065 

0.296 

0.468 

MS 

82.632 

19.894 

85.000 

15.336 

18.500 

-0.328 

0.746 

0.693 

■^Control  19)  -A^Concussed  10. 
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Table  13.  Control  At  Baselines  vs.  Concussed  At  Baseline:  Mean  Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  (Ad) 

meanRT  (SD) 

meanRT  (Ad) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

331.591 

60.778 

315.199 

34.927 

48.598 

1.077 

0.288 

0.705 

RT2 

352.372 

64.683 

325.100 

42.253 

53.780 

1.619 

0.113 

0.187 

CSL 

1191.656 

209.185 

1289.341 

302.398 

263.506 

-1.184 

0.244 

0.278 

PRT 

609.510 

96.054 

640.634 

98.343 

97.293 

-1.021 

0.313 

0.255 

SP 

1637.368 

349.335 

1594.895 

285.083 

316.363 

0.429 

0.671 

0.666 

GNG 

559.446 

88.338 

573.297 

141.875 

120.167 

-0.368 

0.715 

0.705 

CSD 

1044.580 

185.994 

1189.619 

225.441 

208.166 

-2.225 

0.032 

0.025 

MTS 

1599.602 

368.970 

1573.727 

295.950 

331.655 

0.249 

0.805 

0.724 

MS 

880.956 

158.446 

898.428 

228.783 

199.427 

-0.280 

0.781 

0.824 

^Control  19,  ^Concussed  22. 
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Control  8  Days  Post  Baseline  vs.  Concussed  8  Days 

Post  Concussion 


1800 

1600 

1400 

a 

o 

1200 

'■M 

CC  JJ 

o  S 

1000 

cc  -5 

a  h 

800 

rt 

o 

§ 
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RT1 

RT2 

CSL 

PRT 

SP 

GNG 

CSD 

MTS 

MS 

“  control  average  meanRT 

342.191 

352.313 

1107.859 

593.538 

1571.572 

574.451 

1051.137 

1708.672 

842.461 

■  concussed  average  meanRT 

338.555 

336.663 

1220.429 

618.361 

1611.781 

559.424 

1185.015 

1685.681 

854.047 

Table  14.  Control  8  Days  Post  Baseline  vs.  Concussed  8  Days  Post  Concussion:  Mean  Reaction 
Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  ( M) 

meanRT  ( SD ) 

meanRT  ( M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

342.191 

83.778 

338.555 

52.533 

70.868 

0.149 

0.882 

0.523 

RT2 

352.313 

65.231 

336.663 

44.591 

56.503 

0.806 

0.426 

0.629 

CSL 

1107.859 

190.638 

1220.429 

223.772 

206.832 

-1.584 

0.123 

0.133 

PRT 

593.538 

103.429 

618.361 

90.298 

97.495 

-0.741 

0.464 

0.277 

SP 

1571.572 

357.599 

1611.781 

354.023 

355.928 

-0.329 

0.744 

0.641 

GNG 

574.451 

103.566 

559.424 

90.291 

97.568 

0.448 

0.657 

0.769 

CSD 

1051.137 

173.883 

1185.015 

344.811 

267.945 

-1.454 

0.156 

0.479 

MTS 

1708.672 

383.430 

1685.681 

329.130 

359.001 

0.186 

0.853 

1.000 

MS 

842.461 

213.938 

854.047 

259.578 

236.431 

-0.143 

0.887 

0.904 

■^Control  18,  ^Concussed  16. 
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Control  15  Days  Post  Baseline  vs.  Concussed  15 
Days  Post  Concussion 
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RT1 

RT2 

CSLf 

PRT 

SPf 

GNG 

CSD 

MTS 

MS 

“  control  average  meanRT 

353.199 

338.794 

1099.024 

607.054 

1366.796 

569.316 

1098.796 

1641.843 

814.589 

■  concussed  average  meanRT 

344.734 

344.083 

1226.491 

637.196 

1540.886 

578.141 

1192.381 

1771.264 

799.697 

Table  15.  Control  15  Days  Post  Baseline  vs.  Concussed  15  Days  Post  Concussion:  Mean 
Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  ( M) 

meanRT  ( SD ) 

meanRT  ( M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

353.199 

83.496 

344.734 

71.621 

77.785 

0.288 

0.776 

0.872 

RT2 

338.794 

32.738 

344.083 

47.847 

40.994 

-0.341 

0.736 

0.982 

CSL 

1099.024 

144.133 

1226.491 

208.422 

179.184 

-1.882 

0.071 

0.051 

PRT 

607.054 

156.075 

637.196 

154.840 

155.459 

-0.513 

0.612 

0.346 

SP 

1366.796 

248.196 

1540.886 

276.292 

262.620 

-1.754 

0.091 

0.113 

GNG 

569.316 

104.213 

578.141 

111.317 

107.824 

-0.217 

0.830 

0.909 

CSD 

1098.796 

224.184 

1192.381 

255.468 

240.335 

-1.030 

0.312 

0.535 

MTS 

1641.843 

257.690 

1771.264 

408.165 

341.323 

-1.003 

0.325 

0.395 

MS 

814.589 

159.596 

799.697 

218.689 

191.436 

0.206 

0.839 

0.765 

■^Control  14,  -A^Concussed  14. 
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Control  45  Days  Post  Baseline  vs.  Concussed  45 
Days  Post  Concussion 
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RT1 

RT2 

CSL 

PRT 

SP 

GNG 

CSD 

MTS 

MS 

“  control  average  meanRT 

334.105 

344.685 

1186.862 

624.385 

1498.262 

540.605 

1076.867 

1822.546 

821.695 

■  concussed  average  meanRT 

333.873 

349.610 

1223.332 

627.068 

1521.850 

583.790 

1168.890 

1725.830 

886.296 

Table  16.  Control  45  Days  Post  Baseline  vs.  Concussed  45  Days  Post  Concussion:  Mean 
Reaction  Time. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanRT  ( M) 

meanRT  (, SD ) 

meanRT  ( M) 

meanRT  (SD) 

t  (SD) 

t 

P 

P 

RT1 

334.105 

42.655 

333.873 

36.570 

40.160 

0.014 

0.989 

0.975 

RT2 

344.685 

78.532 

349.610 

38.211 

64.420 

-0.182 

0.858 

0.733 

CSL 

1186.862 

213.711 

1223.332 

169.054 

195.823 

-0.443 

0.662 

0.687 

PRT 

624.385 

96.509 

627.068 

106.451 

100.890 

-0.063 

0.950 

0.926 

SP 

1498.262 

335.106 

1521.850 

239.655 

297.967 

-0.188 

0.853 

0.687 

GNG 

540.605 

84.207 

583.790 

95.360 

89.157 

-1.152 

0.262 

0.369 

CSD 

1076.867 

184.891 

1168.890 

293.503 

237.598 

-0.921 

0.368 

0.687 

MTS 

1822.546 

275.655 

1725.830 

269.019 

272.830 

0.843 

0.409 

0.515 

MS 

821.695 

177.843 

886.296 

153.209 

167.729 

-0.916 

0.370 

0.369 

■^Control  1  ^ ,  -A^Concussed  It). 
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Control  Baselines  vs.  Concussed  Baselines 

250 


200 


RT1 

RT2 

CSL 

PRT 

SP 

GNG 

CSD* 

MTS 

MS 

“  control  average  meanThru 

185.608 

172.687 

48.948 

96.287 

35.093 

106.746 

55.879 

36.231 

60.312 

■  concussed  average  meanThru 

191.751 

183.877 

47.334 

92.723 

35.864 

107.535 

46.937 

36.774 

51.870 

Table  17.  Control  At  Baselines  vs.  Concussed  At  Baseline:  Mean  Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThru 

(M) 

meanThru 

(SD) 

meanThru 

(M) 

meanThru 

(SD) 

t(SD ) 

t 

P 

P 

RT1 

185.608 

29.594 

191.751 

19.709 

24.766 

-0.792 

0.433 

0.705 

RT2 

172.687 

30.439 

183.877 

24.805 

27.549 

-1.297 

0.202 

0.245 

CSL 

48.948 

7.085 

47.334 

11.535 

9.737 

0.529 

0.600 

0.505 

PRT 

96.287 

13.929 

92.723 

13.523 

13.712 

0.830 

0.412 

0.395 

SP 

35.093 

7.200 

35.864 

6.453 

6.808 

-0.361 

0.720 

0.629 

GNG 

106.746 

15.947 

107.535 

20.243 

18.385 

-0.137 

0.892 

0.456 

CSD 

55.879 

9.550 

46.937 

9.132 

9.327 

3.061 

0.004 

0.003 

MTS 

36.231 

11.311 

36.774 

7.458 

9.434 

-0.184 

0.855 

0.255 

MS 

60.312 

17.224 

51.870 

18.606 

17.981 

1.499 

0.142 

0.178 

^Control  19,  N Concussed  22. 
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Table  18.  Control  8  Days  Post  Baseline  vs.  Concussed  8  Days  Post  Concussion:  Mean 
Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThru 

(M) 

meanThru 

(SD) 

meanThru 

(M) 

meanThru 

(SD) 

t(SD) 

t 

P 

P 

RT1 

179.256 

32.657 

180.851 

26.936 

30.111 

-0.154 

0.878 

0.593 

RT2 

169.803 

31.585 

180.301 

22.940 

27.869 

-1.096 

0.281 

0.334 

CSL 

53.579 

9.513 

49.114 

9.357 

9.440 

1.376 

0.178 

0.196 

PRT 

99.535 

14.138 

94.307 

12.687 

13.477 

1.129 

0.267 

0.343 

SP 

35.694 

6.107 

34.729 

5.938 

6.028 

0.466 

0.644 

0.617 

GNG 

104.062 

18.034 

108.107 

17.424 

17.751 

-0.663 

0.512 

0.593 

CSD 

53.915 

7.239 

50.396 

13.652 

10.733 

0.954 

0.347 

0.743 

MTS 

33.525 

8.737 

35.154 

7.657 

8.248 

-0.575 

0.569 

0.717 

MS 

62.479 

17.405 

58.429 

19.936 

18.634 

0.633 

0.532 

0.641 

^Control  1  ^ )  ^Concussed  16- 
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Table  19.  Control  15  Days  Post  Baseline  vs.  Concussed  15  Days  Post  Concussion:  Mean 
Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThru 

(M) 

meanThru 

(SD) 

meanThru 

(M) 

meanThru 

(SD) 

t(SD) 

t 

P 

P 

RT1 

175.440 

32.670 

177.161 

29.404 

31.080 

-0.147 

0.885 

0.872 

RT2 

177.824 

18.240 

176.109 

24.952 

21.855 

0.208 

0.837 

0.909 

CSL 

52.633 

5.999 

48.607 

9.416 

7.895 

1.349 

0.189 

0.085 

PRT 

99.491 

19.564 

95.649 

16.743 

18.208 

0.558 

0.581 

0.662 

SP 

40.277 

8.468 

37.267 

6.087 

7.374 

1.080 

0.290 

0.448 

GNG 

105.470 

23.690 

103.678 

20.853 

22.316 

0.212 

0.833 

0.909 

CSD 

51.003 

10.521 

48.372 

9.558 

10.051 

0.693 

0.495 

0.535 

MTS 

34.632 

4.928 

32.250 

8.158 

6.739 

0.935 

0.358 

0.190 

MS 

69.666 

14.139 

63.617 

22.985 

19.082 

0.839 

0.409 

0.535 

^Control  14,  -A^Concussed  14. 
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Table  20.  Control  45  Days  Post  Baseline  vs.  Concussed  45  Days  Post  Concussion:  Mean 
Throughput. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

meanThru 

(M) 

meanThru 

(SD) 

meanThru 

(M) 

meanThru 

(SD) 

t(SD) 

t 

P 

P 

RT1 

180.388 

22.648 

181.594 

19.232 

21.251 

-0.135 

0.894 

0.780 

RT2 

169.709 

32.349 

170.558 

20.539 

27.906 

-0.072 

0.943 

0.877 

CSL 

50.354 

8.904 

48.908 

7.117 

8.186 

0.420 

0.679 

0.828 

PRT 

94.250 

15.107 

96.188 

13.584 

14.474 

-0.318 

0.753 

0.642 

SP 

38.212 

9.430 

38.343 

6.220 

8.209 

-0.038 

0.970 

0.975 

GNG 

111.572 

16.964 

105.007 

17.569 

17.226 

0.906 

0.375 

0.476 

CSD 

53.646 

8.388 

49.597 

13.770 

11.021 

0.873 

0.392 

0.687 

MTS 

31.560 

5.775 

33.773 

6.763 

6.218 

-0.846 

0.407 

0.369 

MS 

67.100 

20.340 

62.491 

19.421 

19.951 

0.549 

0.589 

0.598 

^Control  1 3 ,  -A^Concussed  It). 
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Table  21.  Control  At  Baselines  vs.  Concussed  At  Baseline:  Percent  Correct. 


Control 

Concussed 

T-test  Results 

Wilcoxon 

pctCor  ( M) 

pctCor  (SD) 

pctCor  ( M) 

pctCor  (SD) 

t(SD ) 

t 

P 

P 

RT1 

99.605 

1.254 

99.432 

1.321 

1.290 

0.429 

0.670 

0.537 

RT2 

97.237 

2.992 

98.636 

3.060 

3.029 

-1.475 

0.148 

0.042 

CSL 

94.444 

3.761 

95.581 

4.488 

4.168 

-0.871 

0.389 

0.090 

PRT 

95.559 

3.346 

97.159 

2.535 

2.937 

-1.739 

0.090 

0.112 

SP 

91.842 

7.305 

91.364 

6.207 

6.736 

0.227 

0.822 

0.715 

GNG 

97.719 

2.734 

98.636 

2.220 

2.471 

-1.185 

0.243 

0.266 

CSD 

92.836 

6.164 

88.763 

7.298 

6.798 

1.913 

0.063 

0.039 

MTS 

87.368 

9.912 

87.727 

8.254 

9.057 

-0.126 

0.900 

0.979 

MS 

82.632 

19.894 

71.212 

23.266 

21.775 

1.675 

0.102 

0.048 

^Control  19)  -A^Concussed  22. 
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Control  8  Days  Post  Baseline  vs.  Concussed  8  Days 

Post  Concussion 
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RT1 

RT2| 

CSL 

PRT 

SP 

GNG 

CSD 

MTS 

MS 

“  control  average  pctCor 

98.333 

96.389 

95.833 

96.528 

90.000 

97.222 

91.667 

84.444 

83.519 

■  concussed  average  pctCor 

99.219 

99.219 

96.094 

95.508 

90.000 

98.750 

90.278 

88.750 

79.375 

Table  22.  Control  8  Days  Post  Baseline  vs.  Concussed  8  Days  Post  Concussion:  Mean 
Throughput. 


Control 

Concussed 

T-test  Results  Wilcoxon 

pctCor  ( M) 

pctCor  (, SD ) 

pctCor  ( M) 

pctCor  (SD) 

t  (SD) 

t 

P 

P 

RT1 

98.333 

3.321 

99.219 

1.760 

2.704 

-0.953 

0.348 

0.643 

RT2 

96.389 

6.314 

99.219 

1.505 

4.716 

-1.746 

0.090 

0.134 

CSL 

95.833 

2.382 

96.094 

3.295 

2.847 

-0.266 

0.792 

0.540 

PRT 

96.528 

3.196 

95.508 

3.948 

3.568 

0.832 

0.412 

0.487 

SP 

90.000 

7.859 

90.000 

7.303 

7.603 

0.000 

1.000 

0.986 

GNG 

97.222 

3.477 

98.750 

1.667 

2.779 

-1.600 

0.119 

0.253 

CSD 

91.667 

8.784 

90.278 

9.185 

8.974 

0.450 

0.655 

0.807 

MTS 

84.444 

11.547 

88.750 

5.821 

9.312 

-1.346 

0.188 

0.328 

MS 

83.519 

17.950 

79.375 

21.815 

19.855 

0.607 

0.548 

0.578 

■^Control  18,  ^Concussed  16. 
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Control  15  Days  Post  Baseline  vs.  Concussed  15 
Days  Post  Concussion 
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u 

RT1 

RT2| 

CSL 

PRT 

SP 

GNG 

CSD 

MTS 

MS 

control  average  pctCor 

97.321 

99.286 

95.238 

95.313 

87.857 

96.429 

87.103 

88.095 

91.667 

■  concussed  average  pctCor 

96.786 

98.393 

96.032 

97.545 

92.143 

96.905 

88.690 

86.905 

84.048 

Table  23.  Control  15  Days  Post  Baseline  vs.  Concussed  15  Days  Post  Concussion:  Mean 
Throughput. 


Control 

Concussed 

T-test  Results  Wilcoxon 

pctCor  ( M) 

pctCor  (SD) 

pctCor  ( M) 

pctCor  (SD) 

t  (SD) 

t 

P 

P 

RT1 

97.321 

4.750 

96.786 

5.409 

5.090 

0.278 

0.783 

0.591 

RT2 

99.286 

2.064 

98.393 

1.862 

1.966 

1.202 

0.240 

0.074 

CSL 

95.238 

4.954 

96.032 

3.307 

4.212 

-0.498 

0.622 

1.000 

PRT 

95.313 

4.707 

97.545 

2.789 

3.869 

-1.526 

0.139 

0.184 

SP 

87.857 

8.926 

92.143 

5.447 

7.394 

-1.534 

0.137 

0.166 

GNG 

96.429 

6.976 

96.905 

6.976 

6.976 

-0.181 

0.858 

0.663 

CSD 

87.103 

9.025 

88.690 

7.585 

8.336 

-0.504 

0.619 

0.746 

MTS 

88.095 

3.125 

86.905 

6.467 

5.079 

0.620 

0.541 

0.867 

MS 

91.667 

16.317 

84.048 

25.726 

21.542 

0.936 

0.358 

0.153 

■^Control  14,  -A^Concussed  14. 
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Control  45  Days  Post  Baseline  vs.  Concussed  45 
Days  Post  Concussion 

100 
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RT1|* 

RT2 

CSL| 

PRT 

SP| 

GNG 

CSD 

MTS 

MS 

control  average  pctCor 

98.654 

94.615 

95.513 

95.673 

89.615 

98.205 

92.521 

87.949 

86.154 

-  concussed  average  pctCor 

100.000 

98.500 

98.055 

98.125 

94.500 

99.667 

88.055 

91.000 

85.000 

Table  24.  Control  45  Days  Post  Baseline  vs.  Concussed  45  Days  Post  Concussion:  Mean 
Throughput. 


Control 

Concussed 

T-test  Results  Wilcoxon 

pctCor  ( M) 

pctCor  (, SD ) 

pctCor  (M) 

pctCor  (SD) 

t  (SD) 

t 

P 

P 

RT1 

98.654 

2.193 

100.000 

0.000 

1.657 

-1.931 

0.067 

0.034 

RT2 

94.615 

11.173 

98.500 

1.748 

8.523 

-1.084 

0.291 

1.000 

CSL 

95.513 

4.211 

98.055 

1.630 

3.357 

-1.800 

0.086 

0.173 

PRT 

95.673 

4.864 

98.125 

2.185 

3.946 

-1.477 

0.154 

0.263 

SP 

89.615 

7.206 

94.500 

4.378 

6.155 

-1.887 

0.073 

0.074 

GNG 

98.205 

3.225 

99.667 

1.054 

2.533 

-1.372 

0.185 

0.228 

CSD 

92.521 

7.117 

88.055 

13.737 

10.479 

1.013 

0.322 

0.490 

MTS 

87.949 

7.010 

91.000 

5.676 

6.472 

-1.121 

0.275 

0.285 

MS 

86.154 

18.250 

85.000 

15.336 

17.062 

0.161 

0.874 

0.730 

■^Control  1 3 ,  -A^Concussed  It). 


Baseline  (controls)  vs.  24-hour  (concussed) 

meanRT 


Controls  Concussed 


N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

327.866 

55.304 

18 

345.772 

64.197 

0.328 

-0.989 

CSL1 

26 

1207.597 

234.592 

18 

1244.743 

246.673 

0.616 

-0.506 

PRT1 

26 

619.007 

108.165 

18 

625.028 

85.165 

0.845 

-0.197 

SP1 

26 

1706.992 

355.904 

18 

1689.939 

328.314 

0.873 

0.161 

GNG1 

26 

539.972 

63.228 

18 

571.193 

94.926 

0.197 

-1.312 

CSD2 

26 

1074.090 

187.137 

18 

1120.066 

254.494 

0.493 

-0.691 

MTS1 

26 

1667.365 

261.038 

18 

1668.200 

263.130 

0.992 

-0.010 

MSI 

26 

910.263 

220.549 

18 

847.723 

209.511 

0.351 

0.944 

RT2 

26 

345.570 

39.562 

18 

346.808 

52.982 

0.930 

-0.089 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  mean  reaction  time  between  baselines  for  controls  and  24-hour 
assessments  of  concussed  athletes. 


Baseline  (controls)  vs.  8  Day  (concussed) 

meanRT 


Controls  Concussed 


N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

327.866 

55.304 

18 

334.701 

50.630 

0.679 

-0.417 

CSL1 

26 

1207.597 

234.592 

18 

1191.663 

236.205 

0.826 

0.221 

PRT1 

26 

619.007 

108.165 

18 

608.993 

90.577 

0.749 

0.322 

SP1 

26 

1706.992 

355.904 

18 

1576.978 

348.014 

0.236 

1.202 

GNG1 

26 

539.972 

63.228 

18 

546.994 

92.239 

0.766 

-0.300 

CSD2 

26 

1074.090 

187.137 

18 

1147.413 

341.915 

0.365 

-0.916 

MTS1 

26 

1667.365 

261.038 

18 

1657.517 

320.309 

0.911 

0.112 

MSI 

26 

910.263 

220.549 

18 

837.283 

248.673 

0.312 

1.024 

RT2 

26 

345.570 

39.562 

18 

333.846 

42.682 

0.355 

0.936 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  mean  reaction  time  between  baselines  for  controls  and 
assessments  8  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  15  Day  (concussed) 


meanRT 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

327.866 

55.304 

18 

333.931 

67.574 

0.746 

-0.327 

CSL1 

26 

1207.597 

234.592 

18 

1176.234 

217.354 

0.656 

0.449 

PRT1 

26 

619.007 

108.165 

18 

612.132 

147.672 

0.859 

0.178 

SP1 

26 

1706.992 

355.904 

18 

1490.531 

300.871 

0.041 

2.109 

GNG1 

26 

539.972 

63.228 

18 

563.906 

106.625 

0.356 

-0.934 

CSD2 

26 

1074.090 

187.137 

18 

1133.834 

256.596 

0.376 

-0.894 

MTS1 

26 

1667.365 

261.038 

18 

1729.706 

375.693 

0.519 

-0.650 

MSI 

26 

910.263 

220.549 

18 

797.416 

202.616 

0.092 

1.724 

RT2 

26 

345.570 

39.562 

18 

337.914 

45.576 

0.556 

0.593 

There  are  significant  differences  mean  reaction  time  on  Spatial  Processing  between  baselines  for  controls  and  15  days  after 
concussion  for  concussed  athletes.  No  other  differences  are  significant  for  the  remaining  DANA  subtests  for  mean  reaction  time 
between  baselines  for  controls  and  assessments  15  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  45  Day  (concussed) 


meanRT 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

327.866 

55.304 

14 

329.468 

47.442 

0.927 

-0.092 

CSL1 

26 

1207.597 

234.592 

14 

1190.376 

190.091 

0.815 

0.236 

PRT1 

26 

619.007 

108.165 

14 

600.013 

102.247 

0.593 

0.540 

SP1 

26 

1706.992 

355.904 

14 

1500.196 

270.644 

0.066 

1.895 

GNG1 

26 

539.972 

63.228 

14 

564.184 

94.814 

0.340 

-0.967 

CSD2 

26 

1074.090 

187.137 

14 

1134.795 

280.372 

0.418 

-0.820 

MTS1 

26 

1667.365 

261.038 

14 

1664.471 

275.309 

0.974 

0.033 

MSI 

26 

910.263 

220.549 

14 

831.840 

155.946 

0.246 

1.178 

RT2 

26 

345.570 

39.562 

14 

344.289 

57.107 

0.934 

0.083 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  mean  reaction  time  between  baselines  for  controls  and 
assessments  45  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  24-hour  (concussed) 

meanThru 


Controls  Concussed 


N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

186.800 

25.543 

18 

178.282 

27.421 

0.297 

1.056 

CSL1 

26 

49.216 

9.242 

18 

47.924 

8.563 

0.641 

0.470 

PRT1 

26 

96.388 

14.054 

18 

94.673 

12.453 

0.679 

0.416 

SP1 

26 

34.459 

6.004 

18 

33.529 

6.664 

0.632 

0.483 

GNG1 

26 

105.607 

24.883 

18 

106.890 

17.029 

0.850 

-0.190 

CSD2 

26 

53.694 

8.714 

18 

52.811 

10.568 

0.764 

0.303 

MTS1 

26 

35.120 

6.762 

18 

34.643 

6.187 

0.813 

0.238 

MSI 

26 

56.605 

16.976 

18 

55.454 

23.374 

0.851 

0.190 

RT2 

26 

174.781 

20.089 

18 

174.482 

25.214 

0.965 

0.044 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  mean  throughput  between  baselines  for  controls  and  24-hour 
assessments  of  concussed  athletes. 


Baseline  (controls)  vs.  8  Day  (concussed) 
meanThru 


Controls  Concussed 


N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

186.800 

25.543 

18 

182.702 

25.889 

0.606 

0.520 

CSL1 

26 

49.216 

9.242 

18 

50.606 

10.593 

0.647 

-0.462 

PRT1 

26 

96.388 

14.054 

18 

96.192 

13.411 

0.963 

0.046 

SP1 

26 

34.459 

6.004 

18 

35.436 

5.953 

0.597 

-0.532 

GNG1 

26 

105.607 

24.883 

18 

110.632 

17.945 

0.467 

-0.734 

CSD2 

26 

53.694 

8.714 

18 

51.790 

13.548 

0.573 

0.568 

MTS1 

26 

35.120 

6.762 

18 

35.811 

7.516 

0.752 

-0.318 

MSI 

26 

56.605 

16.976 

18 

61.062 

20.279 

0.434 

-0.791 

RT2 

26 

174.781 

20.089 

18 

181.241 

21.750 

0.316 

-1.014 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  mean  throughput  between  baselines  for  controls  and 
assessments  8  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  15  Day  (concussed) 


meanThru 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

186.800 

25.543 

18 

183.251 

29.762 

0.674 

0.423 

CSL1 

26 

49.216 

9.242 

18 

50.801 

9.945 

0.591 

-0.542 

PRT1 

26 

96.388 

14.054 

18 

99.989 

18.103 

0.462 

-0.743 

SP1 

26 

34.459 

6.004 

18 

39.126 

8.183 

0.035 

-2.184 

GNG1 

26 

105.607 

24.883 

18 

106.720 

20.213 

0.876 

-0.157 

CSD2 

26 

53.694 

8.714 

18 

51.623 

11.365 

0.498 

0.684 

MTS1 

26 

35.120 

6.762 

18 

33.022 

7.547 

0.340 

0.965 

MSI 

26 

56.605 

16.976 

18 

66.251 

21.373 

0.103 

-1.666 

RT2 

26 

174.781 

20.089 

18 

178.978 

24.490 

0.537 

-0.623 

There  are  significant  differences  mean  throughput  on  Spatial  Processing  between  baselines  for  controls  and  15  days  after  concussion 
for  concussed  athletes.  No  other  differences  are  significant  for  the  remaining  DANA  subtests  for  mean  throughput  between  baselines 
for  controls  and  assessments  15  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  45  Day  (concussed) 


meanThru 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

186.800 

25.543 

14 

184.561 

25.779 

0.794 

0.263 

CSL1 

26 

49.216 

9.242 

14 

49.871 

8.017 

0.824 

-0.223 

PRT1 

26 

96.388 

14.054 

14 

99.753 

13.793 

0.472 

-0.727 

SP1 

26 

34.459 

6.004 

14 

38.857 

6.327 

0.036 

-2.169 

GNG1 

26 

105.607 

24.883 

14 

108.262 

17.968 

0.727 

-0.352 

CSD2 

26 

53.694 

8.714 

14 

51.341 

13.367 

0.505 

0.673 

MTS1 

26 

35.120 

6.762 

14 

34.589 

6.895 

0.815 

0.235 

MSI 

26 

56.605 

16.976 

14 

64.505 

23.139 

0.225 

-1.234 

RT2 

26 

174.781 

20.089 

14 

175.001 

27.780 

0.977 

-0.029 

There  are  significant  differences  mean  throughput  on  Spatial  Processing  between  baselines  for  controls  and  45  days  after  concussion 
for  concussed  athletes.  No  other  differences  are  significant  for  the  remaining  DANA  subtests  for  mean  throughput  between  baselines 
for  controls  and  assessments  45  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  24-hour  (concussed) 


percentCorrect 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

99.423 

2.481 

18 

98.611 

2.604 

0.301 

1.046 

CSL1 

26 

95.299 

3.493 

18 

95.602 

3.993 

0.791 

-0.266 

PRT1 

26 

96.755 

3.901 

18 

97.049 

2.507 

0.780 

-0.281 

SP1 

26 

94.231 

5.233 

18 

90.556 

8.382 

0.080 

1.792 

GNG1 

26 

94.872 

19.487 

18 

99.445 

1.278 

0.328 

-0.990 

CSD2 

26 

92.628 

5.876 

18 

91.512 

4.994 

0.515 

0.657 

MTS1 

26 

89.487 

6.160 

18 

88.704 

6.968 

0.696 

0.393 

MSI 

26 

80.256 

20.932 

18 

74.259 

30.137 

0.440 

0.780 

RT2 

26 

98.269 

2.896 

18 

97.361 

5.654 

0.488 

0.699 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  percent  correct  between  baselines  for  controls  and  24-hour 
assessments  of  concussed  athletes. 


Baseline  (controls)  vs.  8  Day  (concussed) 


percentCorrect 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

99.423 

2.481 

18 

99.306 

1.673 

0.862 

0.175 

CSL1 

26 

95.299 

3.493 

18 

96.142 

3.180 

0.419 

-0.816 

PRT1 

26 

96.755 

3.901 

18 

95.833 

3.865 

0.444 

0.773 

SP1 

26 

94.231 

5.233 

18 

89.722 

6.960 

0.018 

2.454 

GNG1 

26 

94.872 

19.487 

18 

98.704 

1.672 

0.412 

-0.829 

CSD2 

26 

92.628 

5.876 

18 

89.969 

9.293 

0.251 

1.164 

MTS1 

26 

89.487 

6.160 

18 

89.074 

5.578 

0.821 

0.227 

MSI 

26 

80.256 

20.932 

18 

80.926 

21.107 

0.918 

-0.104 

RT2 

26 

98.269 

2.896 

18 

99.167 

1.485 

0.234 

-1.207 

There  are  significant  differences  percent  correct  on  Spatial  Processing  between  baselines  for  controls  and  8  days  after  concussion  for 
concussed  athletes.  No  other  differences  are  significant  for  the  remaining  DANA  subtests  for  percent  correct  between  baselines  for 
controls  and  assessments  8  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  15  Day  (concussed) 


percentCorrect 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

99.423 

2.481 

18 

97.500 

4.926 

0.095 

1.708 

CSL1 

26 

95.299 

3.493 

18 

96.065 

3.097 

0.459 

-0.748 

PRT1 

26 

96.755 

3.901 

18 

97.917 

2.626 

0.277 

-1.101 

SP1 

26 

94.231 

5.233 

18 

92.500 

4.926 

0.276 

1.104 

GNG1 

26 

94.872 

19.487 

18 

97.408 

6.216 

0.598 

-0.532 

CSD2 

26 

92.628 

5.876 

18 

90.278 

7.395 

0.247 

1.173 

MTS1 

26 

89.487 

6.160 

18 

86.667 

5.717 

0.132 

1.537 

MSI 

26 

80.256 

20.932 

18 

86.667 

23.066 

0.344 

-0.958 

RT2 

26 

98.269 

2.896 

18 

98.611 

1.762 

0.658 

-0.446 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  percent  correct  between  baselines  for  controls  and  assessments 
15  days  after  concussion  for  concussed  athletes. 


Baseline  (controls)  vs.  45  Day  (concussed) 


percentCorrect 


Controls 

Concussed 

N 

M 

SD 

N 

M 

SD 

P 

t 

RT1 

26 

99.423 

2.481 

14 

99.464 

1.447 

0.955 

-0.057 

CSL1 

26 

95.299 

3.493 

14 

96.925 

2.840 

0.144 

-1.493 

PRT1 

26 

96.755 

3.901 

14 

97.545 

2.789 

0.507 

-0.669 

SP1 

26 

94.231 

5.233 

14 

93.929 

5.942 

0.869 

0.166 

GNG1 

26 

94.872 

19.487 

14 

99.286 

1.419 

0.405 

-0.841 

CSD2 

26 

92.628 

5.876 

14 

88.492 

11.471 

0.138 

1.516 

MTS1 

26 

89.487 

6.160 

14 

90.000 

7.845 

0.821 

-0.228 

MSI 

26 

80.256 

20.932 

14 

82.143 

22.517 

0.793 

-0.265 

RT2 

26 

98.269 

2.896 

14 

97.857 

2.373 

0.651 

0.456 

There  are  no  significant  differences  in  any  of  the  DANA  subtests  for  percent  correct  between  baselines  for  controls  and  assessments 
45  days  after  concussion  for  concussed  athletes. 


Appendix  O:  ECT  Statistics 


patientName  MMSE  Score 

TestDate 

1 

26 

1/13/14 

1 

25 

1/14/14 

1 

20 

1/23/14 

1 

25 

1/27/14 

1 

22 

1/28/14 

1 

19 

2/3/14 

1 

23 

2/4/14 

1 

22 

2/10/14 

1 

22 

2/11/14 

2 

29 

1/22/14 

2 

29 

1/28/14 

2 

28 

2/18/14 

2 

30 

2/24/14 

2 

29 

3/5/14 

3 

25 

2/4/14 

3 

27 

2/10/14 

3 

30 

2/11/14 

3 

30 

2/17/14 

3 

29 

2/18/14 

3 

29 

3/10/14 

3 

27 

3/21/14 

3 

30 

3/24/14 

3 

29 

3/25/14 

3 

27 

3/31/14 

3 

30 

4/1/14 

3 

28 

4/9/14 

3 

28 

4/14/14 

3 

29 

4/15/14 

3 

29 

4/21/14 

3 

27 

4/22/14 

3 

27 

5/5/14 

3 

26 

6/19/14 

3 

29 

7/2/14 

3 

28 

7/9/14 

3 

29 

7/23/14 

5 

28 

2/11/14 

5 

30 

2/18/14 

5 

30 

2/24/14 

5 

30 

2/25/14 

6 

30 

2/6/14 

6 

30 

2/10/14 

6 

29 

2/11/14 

10 

30 

3/20/14 

10 

30 

3/25/14 

10 

30 

3/31/14 

10 

30 

4/1/14 

10 

29 

4/9/14 

10 

30 

4/10/14 

10 

30 

4/15/14 

10 

30 

4/22/14 

meanRT(RT2)  NumCorrect(RT2)  meanThru(RT2) 


545.2 

24 

108.59 

639.28 

23 

89.522 

604.04 

19 

82.428 

699.6 

17 

62.441 

630.08 

21 

81.673 

590.68 

18 

82.997 

633.84 

20 

84.611 

535.48 

23 

109.57 

556.08 

24 

106.32 

306.2 

25 

195.95 

322.84 

25 

185.85 

348.24 

25 

172.29 

350.84 

25 

171.02 

336.4 

25 

178.36 

602.76 

24 

97.565 

423.92 

25 

141.54 

479.68 

25 

125.08 

641.32 

17 

78.524 

537.2 

22 

108.26 

581.72 

20 

95.589 

479.12 

24 

124.79 

467.6 

24 

128.12 

420.64 

25 

142.64 

436.16 

25 

137.56 

627.56 

19 

84.206 

584.32 

21 

96.148 

591.56 

19 

92.278 

559.44 

20 

101.2 

503.68 

23 

117.64 

689.68 

17 

69.07 

536.92 

23 

109.23 

525.32 

23 

112.03 

629.52 

22 

89.093 

408.4 

24 

137.2 

559 

21 

102.01 

432.88 

25 

138.61 

413.88 

25 

144.97 

435.64 

24 

138.36 

507.12 

25 

118.32 

414.52 

23 

148.27 

355.56 

25 

168.75 

367.44 

25 

163.29 

403.12 

24 

150.62 

423.08 

25 

141.82 

385.04 

25 

155.83 

405.08 

25 

148.12 

388 

25 

154.64 

437.4 

25 

137.17 

419.96 

25 

142.87 

365.72 

25 

164.06 

DANA  vs.  MMSE 


meanRT(RT2) 

_  900 
j=  800 
g  700 

H  600 

|  500 

S  400 

“  300 

S  200 
^  100 
a:  o 

10  12  14  16  18  20  22  24  26  28  30  32  34  36  38  40 

MMSE  Score 


NumCorrect(RT2) 


MMSE  Score 


- Linear  Forecast  Trendline 


meanThru(RT2) 


MMSE  Score 


- Linear  Forecast  Trendline 


10 

30 

4/24/14 

399.96 

24 

138.84 

14 

27 

7/29/14 

738.4 

22 

73.706 

19 

29 

6/5/14 

526.08 

25 

114.05 

19 

30 

6/10/14 

475.32 

25 

126.23 

19 

30 

6/17/14 

525.92 

25 

114.09 

20 

28 

6/4/14 

394.32 

25 

152.16 

20 

28 

6/11/14 

412.96 

25 

145.29 

20 

26 

6/13/14 

430.08 

25 

139.51 

20 

27 

6/16/14 

488.8 

25 

122.75 

20 

29 

7/9/14 

486.04 

23 

122.65 

21 

30 

7/28/14 

323.04 

25 

185.74 

21 

30 

8/5/14 

367.88 

25 

163.1 

21 

30 

8/11/14 

308.2 

24 

179.67 

21 

30 

8/12/14 

320.28 

25 

187.34 

21 

30 

8/18/14 

330.4 

25 

181.6 

21 

30 

8/19/14 

345.32 

25 

173.75 

21 

30 

8/21/14 

375.8 

25 

159.66 

21 

30 

8/25/14 

387.56 

25 

154.81 

21 

30 

8/26/14 

370.68 

25 

161.86 

21 

29 

8/28/14 

381.44 

24 

145.39 

21 

30 

9/4/14 

388.56 

23 

146.22 

21 

27 

9/9/14 

380.24 

25 

157.8 

21 

30 

9/11/14 

381.52 

25 

157.27 

22 

30 

7/29/14 

453.8 

24 

132.35 

23 

30 

7/24/14 

571.04 

23 

101.76 

23 

28 

7/29/14 

582.64 

23 

99.452 

24 

30 

8/18/14 

578 

24 

102.02 

24 

29 

8/19/14 

664.84 

21 

81.284 

24 

30 

8/21/14 

546.88 

22 

105.87 

25 

30 

7/31/14 

403.16 

24 

150.6 

25 

30 

8/5/14 

464.96 

23 

129.23 

25 

30 

8/7/14 

491.04 

25 

122.19 

25 

30 

8/12/14 

572.84 

23 

101.4 

25 

30 

8/14/14 

425.6 

25 

140.98 

25 

30 

8/19/14 

452.48 

24 

132.77 

25 

30 

8/21/14 

614.36 

20 

88.406 

25 

29 

8/25/14 

407.28 

25 

147.32 

27 

29 

8/5/14 

790.96 

21 

65.438 

27 

29 

8/7/14 

548.12 

25 

109.47 

27 

30 

8/12/14 

614.36 

24 

95.608 

27 

29 

8/18/14 

675.4 

24 

86.481 

27 

30 

8/26/14 

586.64 

25 

102.28 

27 

27 

8/28/14 

738.12 

20 

68.802 

27 

28 

9/2/14 

708.24 

19 

70.405 

27 

29 

9/5/14 

709.4 

24 

82.115 

27 

30 

9/9/14 

618.6 

25 

96.993 

27 

21 

9/18/14 

654.28 

25 

91.704 

27 

27 

9/23/14 

693.4 

23 

81.725 

27 

25 

9/25/14 

722.68 

18 

66.083 

27 

28 

9/29/14 

698.56 

20 

74.051 

27 

30 

10/7/14 

667.64 

22 

83.025 

28 

29 

8/13/14 

584.24 

22 

97.564 

28 

29 

8/18/14 

635.6 

24 

92.221 

28 

27 

8/19/14 

544.6 

24 

108.72 

28 

24 

8/26/14 

672.36 

17 

72.182 

28 

25 

8/28/14 

643.96 

25 

93.173 

28 

26 

9/2/14 

576.2 

23 

100.72 

28 

27 

9/3/14 

558.48 

22 

103.14 

29 

30 

8/19/14 

497.72 

25 

120.55 

29 

30 

8/21/14 

553.88 

24 

106.77 

29 

30 

8/25/14 

462.48 

25 

129.74 

29 

30 

8/28/14 

545.56 

20 

105.04 

29 

30 

9/2/14 

603.32 

21 

92.171 

29 

30 

9/11/14 

424.04 

24 

142.5 

29 

29 

9/15/14 

514 

25 

116.73 

29 

30 

9/16/14 

395.48 

25 

151.71 

32 

30 

8/20/14 

578.12 

25 

103.78 

32 

30 

8/28/14 

601.8 

22 

94.095 

32 

30 

9/2/14 

392.92 

25 

152.7 

32 

28 

9/3/14 

452.92 

25 

132.47 

32 

27 

9/4/14 

493.44 

25 

121.6 

33 

29 

8/26/14 

415.56 

23 

134.01 

33 

30 

8/28/14 

391.04 

24 

143.03 

33 

30 

9/3/14 

473.84 

23 

126.38 

33 

29 

9/4/14 

430.24 

24 

140.26 

patientName  MMSE  Score 


1 

26 

1 

25 

1 

25 

1 

22 

1 

19 

1 

23 

1 

22 

1 

22 

2 

29 

2 

29 

2 

28 

2 

30 

2 

29 

3 

25 

3 

27 

3 

30 

3 

30 

3 

29 

3 

29 

3 

27 

3 

30 

3 

29 

3 

27 

3 

28 

3 

27 

3 

28 

3 

29 

3 

29 

3 

27 

3 

27 

3 

26 

3 

29 

3 

28 

3 

28 

3 

29 

5 

28 

5 

30 

5 

30 

5 

30 

6 

30 

6 

30 

6 

29 

10 

30 

10 

30 

10 

30 

10 

30 

10 

29 

10 

30 

10 

30 

10 

30 

10 

30 

14 

27 

19 

29 

19 

30 

19 

30 

;  meanRT(RTl) 

1/13/14 
1/14/14 
1/27/14 
1/28/14 
2/3/14 
2/4/14 
2/10/14 
2/11/14 
1/22/14 
1/28/14 
2/18/14 
2/24/14 
3/5/14 
2/4/14 
2/10/14 
2/11/14 
2/17/14 
2/18/14 
3/10/14 
3/21/14 
3/24/14 
3/25/14 
3/31/14 
4/9/14 
4/10/14 
4/14/14 
4/15/14 
4/21/14 
4/22/14 
5/5/14 
6/19/14 
7/2/14 
7/9/14 
7/18/14 
7/23/14 
2/11/14 
2/18/14 
2/24/14 
2/25/14 
2/6/14 
2/10/14 
2/11/14 
3/20/14 
3/25/14 
3/31/14 
4/1/14 
4/9/14 
4/10/14 
4/15/14 
4/22/14 
4/24/14 
7/29/14 
6/5/14 
6/10/14 
6/17/14 


NumCorrect(RTl) 

549.8 

511.16 

698.52 

643.36 

644.92 

603.52 

566.12 

535.88 
321.04 

314.48 

327.36 
336.44 

330.4 

642.48 
399.24 

532.2 

425.28 
444 

417.32 

411.12 

377.92 

375.2 

425.84 

467.6 

392.6 
476 

533.48 
560 

747.68 

520.64 

426.92 

472.72 

476.56 

416.28 

366.12 
427.04 

347.92 

335.84 

493.32 
417.96 

377.68 

381.12 

355.72 

395.64 

351.16 

387.48 

387.36 

461.88 

400.2 

372.52 

416.84 

652.68 
476 

447.76 

556.56 


meanThru(RTl) 


24 

107.62 

25 

117.38 

23 

81.057 

21 

84.781 

19 

80.798 

21 

92.131 

23 

102.78 

25 

111.97 

25 

186.89 

25 

190.79 

25 

183.28 

25 

178.34 

25 

181.6 

18 

79.656 

25 

150.29 

21 

109.06 

23 

130.46 

25 

135.14 

24 

145.01 

24 

147.41 

23 

151.16 

25 

159.91 

25 

140.9 

23 

128.37 

25 

152.83 

23 

125.7 

20 

108.63 

19 

100.74 

17 

60.355 

23 

113.2 

23 

129.63 

24 

126.62 

24 

125.51 

24 

145.41 

25 

163.88 

25 

140.5 

25 

172.45 

25 

178.66 

23 

120.54 

22 

149.9 

25 

158.86 

25 

157.43 

25 

168.67 

25 

151.65 

25 

170.86 

25 

154.85 

24 

157.38 

25 

129.9 

25 

149.93 

25 

161.07 

25 

143.94 

23 

87.456 

25 

126.05 

25 

134 

19 

101.76 

250 

200 

150 

100 

50 

0 

10  12  14  16  18  20  22  24  26  28  30  32  34  36  38  40 

MMSE  Score 


- Linear  Forecast  Trendline 


20 

28 

6/4/14 

409.04 

25 

146.68 

20 

28 

6/11/14 

579.8 

18 

94.887 

20 

27 

6/16/14 

536.44 

24 

110.49 

20 

29 

7/9/14 

582.8 

22 

97.86 

21 

30 

7/28/14 

326.32 

25 

183.87 

21 

30 

8/5/14 

335.52 

25 

178.83 

21 

30 

8/11/14 

356.52 

25 

168.29 

21 

30 

8/12/14 

380.12 

24 

160.69 

21 

30 

8/18/14 

344.88 

25 

173.97 

21 

30 

8/19/14 

345.96 

25 

173.43 

21 

30 

8/21/14 

376.92 

24 

162.2 

21 

30 

8/25/14 

367.36 

23 

153.33 

21 

30 

8/26/14 

457.08 

24 

131.32 

21 

29 

8/28/14 

426.72 

24 

141.52 

21 

30 

9/4/14 

374.92 

24 

163.15 

21 

27 

9/9/14 

382.48 

23 

146.47 

21 

30 

9/11/14 

407.16 

23 

136.53 

22 

30 

7/29/14 

390.68 

24 

142.24 

23 

27 

7/9/14 

728.36 

17 

63.003 

23 

30 

7/24/14 

522.28 

22 

112.16 

23 

28 

7/29/14 

604.04 

21 

92.027 

24 

30 

8/18/14 

618 

21 

89.316 

24 

29 

8/19/14 

642.6 

21 

84.91 

24 

30 

8/21/14 

602.6 

24 

97.593 

25 

30 

7/31/14 

367.84 

25 

163.11 

25 

30 

8/5/14 

456.76 

24 

131.42 

25 

30 

8/7/14 

412.08 

25 

145.6 

25 

30 

8/12/14 

423.52 

24 

142.69 

25 

30 

8/14/14 

416.52 

24 

145.32 

25 

30 

8/19/14 

584.28 

21 

89.68 

25 

30 

8/21/14 

455.04 

23 

121.55 

25 

29 

8/25/14 

416.36 

25 

144.11 

27 

29 

8/5/14 

696.72 

25 

86.118 

27 

29 

8/7/14 

588.96 

25 

101.87 

27 

30 

8/12/14 

532.28 

25 

112.72 

27 

29 

8/18/14 

694.92 

21 

76.846 

27 

28 

8/19/14 

533 

25 

112.57 

27 

28 

8/22/14 

643.24 

22 

86.81 

27 

30 

8/26/14 

675.52 

24 

86.465 

27 

27 

8/28/14 

733.8 

24 

79.243 

27 

28 

9/2/14 

662.08 

23 

86.063 

27 

29 

9/5/14 

738.68 

21 

71.191 

27 

30 

9/9/14 

674.84 

25 

88.91 

27 

21 

9/18/14 

677.44 

25 

88.569 

27 

27 

9/23/14 

749.36 

22 

72.446 

27 

25 

9/25/14 

743.96 

17 

60.847 

27 

28 

9/29/14 

744.08 

21 

70.551 

27 

30 

10/7/14 

761.56 

24 

76.211 

28 

29 

8/13/14 

478.64 

23 

124.89 

28 

27 

8/19/14 

613.52 

19 

87.181 

28 

27 

8/21/14 

635.76 

20 

84.255 

28 

24 

8/26/14 

589.56 

22 

96.486 

28 

25 

8/28/14 

596.36 

21 

93.589 

28 

26 

9/2/14 

560.8 

23 

103.9 

28 

27 

9/3/14 

514.96 

22 

114.17 

29 

30 

8/19/14 

534.6 

25 

112.23 

29 

30 

8/21/14 

543.2 

24 

109.02 

29 

30 

8/25/14 

472.88 

25 

126.88 

29 

30 

8/28/14 

485.76 

23 

122.74 

29 

30 

9/2/14 

520.88 

22 

112.54 

29 

30 

9/11/14 

391.04 

25 

153.44 

29 

29 

9/15/14 

459.96 

24 

130.43 

29 

30 

9/16/14 

428.92 

24 

140.73 

32 

30 

8/20/14 

561.72 

24 

105.18 

32 

30 

8/28/14 

440.36 

24 

136.75 

32 

30 

9/2/14 

415.4 

25 

144.44 

32 

28 

9/3/14 

483.96 

24 

123.44 

32 

27 

9/4/14 

420.52 

25 

142.68 

33 

29 

8/26/14 

350.04 

25 

171.41 

33 

30 

8/28/14 

365.76 

25 

164.04 

33 

30 

9/3/14 

402.48 

25 

149.08 

33 

29 

9/4/14 

405.52 

25 

147.96 

Mininum  314.48 

Maximum  761.56 


patientName  MMSE 
1 
1 
1 
1 
1 
1 
1 
1 
1 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
3 
5 
5 
5 

5 

6 
6 
6 

10 

10 

10 

10 

10 

10 


Score  TestDate 


26 

1/13/14 

25 

1/14/14 

20 

1/23/14 

25 

1/27/14 

22 

1/28/14 

19 

2/3/14 

23 

2/4/14 

22 

2/10/14 

22 

2/11/14 

29 

1/22/14 

29 

1/28/14 

28 

2/18/14 

30 

2/24/14 

29 

3/5/14 

25 

2/4/14 

27 

2/10/14 

30 

2/11/14 

30 

2/17/14 

29 

2/18/14 

29 

3/10/14 

27 

3/21/14 

30 

3/24/14 

29 

3/25/14 

27 

3/31/14 

30 

4/1/14 

28 

4/9/14 

27 

4/10/14 

28 

4/14/14 

29 

4/15/14 

29 

4/21/14 

27 

4/22/14 

27 

5/5/14 

26 

6/19/14 

29 

7/2/14 

28 

7/9/14 

28 

7/18/14 

29 

7/23/14 

28 

2/11/14 

30 

2/18/14 

30 

2/24/14 

30 

2/25/14 

30 

2/6/14 

30 

2/10/14 

29 

2/11/14 

30 

3/20/14 

30 

3/25/14 

30 

3/31/14 

30 

4/1/14 

29 

4/9/14 

30 

4/10/14 

meanRT(PRTl) 

881.44 
922.03 

1135.5 

1113.6 

984.81 
1223.4 

1386.7 
1046.9 

870.75 

742.38 

622.38 
616.97 
678.06 
682.31 

808.44 

775.94 

652.94 

731.78 
721.69 
717.06 

702.53 

671.66 

727.66 

808.66 

1070.8 
823.06 
860.59 

737.78 
786 

1026.9 

757.53 
1029.8 
1127.1 

781.75 

930.81 
781.19 
997.72 

702 

735.91 
685.62 

740.91 
830.41 

833.22 

747.22 
666.25 

693.66 

732.75 

696.66 
671.5 

773.34 


NumCorrect(PRTl) 

32 

30 

31 

31 
29 

29 
26 
28 

32 

30 
32 
32 
32 
32 
32 

31 
29 

31 

32 

31 

32 

31 

32 
32 

29 
31 

30 

31 
31 
31 

30 
29 
29 

31 

31 

32 
32 

31 

32 
32 
32 
31 
31 

31 

32 
32 
32 
32 
32 
32 


meanThru(PRTl) 

68.071 

62.576 

52.478 

53.573 

58.286 

44.236 

36.696 

52.973 

68.906 

80.231 

96.405 

97.25 

88.487 

87.936 

74.217 

74.182 

82.421 

78.438 

83.138 

81.063 

85.405 
85.885 
82.457 
74.197 
51.157 
70.361 
67.899 

78.38 

74.76 

58.389 

76.406 
52.598 
52.444 
75.588 
63.321 
76.806 
60.137 
81.848 
81.532 
87.511 
80.982 
73.327 

73.06 

76.905 

90.056 

86.498 

81.883 

86.126 

89.352 

77.585 


meanRT(PRTl) 


MMSE  Score 


- Linear  Forecast  Trendline 


NumCorrect(PRTl) 


MMSE  Score 


meanThru(PRTl) 


MMSE  Score 


- Linear  Forecast  Trendline 


10 

30 

4/15/14 

769.69 

31 

74.779 

10 

30 

4/22/14 

760.19 

32 

78.928 

10 

30 

4/24/14 

769.16 

32 

78.008 

14 

27 

7/29/14 

1296.7 

32 

46.271 

19 

29 

6/5/14 

803.25 

32 

74.697 

19 

30 

6/10/14 

759.47 

32 

79.003 

19 

30 

6/17/14 

950.22 

30 

59.113 

20 

28 

6/4/14 

888.31 

32 

67.544 

20 

28 

6/11/14 

1006.6 

31 

59.645 

20 

26 

6/13/14 

957.69 

32 

62.651 

20 

27 

6/16/14 

1116.6 

32 

53.735 

20 

29 

7/9/14 

990.91 

32 

60.551 

21 

30 

7/28/14 

535.09 

31 

108.04 

21 

30 

8/5/14 

565.28 

32 

106.14 

21 

30 

8/11/14 

653.5 

32 

91.813 

21 

30 

8/12/14 

615.84 

31 

93.672 

21 

30 

8/18/14 

700.5 

32 

85.653 

21 

30 

8/19/14 

808.22 

32 

74.237 

21 

30 

8/21/14 

744.91 

30 

73.417 

21 

30 

8/25/14 

712.09 

32 

84.259 

21 

30 

8/26/14 

757.47 

32 

79.211 

21 

29 

8/28/14 

708.53 

32 

84.682 

21 

30 

9/4/14 

738.31 

32 

81.266 

21 

27 

9/9/14 

770.56 

31 

75.361 

21 

30 

9/11/14 

805.31 

32 

74.505 

22 

30 

7/29/14 

661.66 

32 

90.682 

23 

27 

7/9/14 

1424.4 

29 

39.838 

23 

30 

7/24/14 

1089.2 

29 

51.591 

23 

28 

7/29/14 

1098.3 

29 

50.617 

24 

30 

8/18/14 

893.25 

31 

65.222 

24 

29 

8/19/14 

1256.3 

30 

46.615 

24 

30 

8/21/14 

998.69 

26 

55.054 

25 

30 

7/31/14 

654.88 

31 

88.275 

25 

30 

8/5/14 

658.62 

31 

88.916 

25 

30 

8/7/14 

694.72 

32 

86.366 

25 

30 

8/12/14 

712.59 

32 

84.199 

25 

30 

8/14/14 

859.03 

31 

67.179 

25 

30 

8/19/14 

819.41 

32 

73.224 

25 

30 

8/21/14 

737.41 

31 

77.922 

25 

29 

8/25/14 

675.06 

32 

88.881 

27 

29 

8/5/14 

1041.5 

32 

57.609 

27 

29 

8/7/14 

1000.2 

32 

59.985 

27 

30 

8/12/14 

959.25 

32 

62.549 

27 

29 

8/18/14 

1085.8 

31 

55.029 

27 

28 

8/19/14 

966.62 

32 

62.072 

27 

28 

8/22/14 

1226 

31 

48.395 

27 

30 

8/26/14 

1215.6 

29 

47.931 

27 

27 

8/28/14 

1061.4 

31 

54.385 

27 

28 

9/2/14 

1060.7 

32 

56.569 

27 

29 

9/5/14 

1059 

32 

56.656 

27 

30 

9/9/14 

945.03 

32 

63.49 

27 

21 

9/18/14 

1175.4 

31 

50.596 

27 

27 

9/23/14 

1008.5 

32 

59.494 

27 

25 

9/25/14 

1144.5 

31 

52.043 

27 

28 

9/29/14 

1115.5 

32 

53.786 

27 

30 

10/7/14 

1239.6 

27 

46.072 

28 

29 

8/13/14 

1230.7 

27 

43.855 

28 

29 

8/18/14 

1127.9 

32 

53.196 

28 

27 

8/19/14 

1256.9 

32 

47.735 

28 

27 

8/21/14 

1109.5 

32 

54.08 

28 

24 

8/26/14 

1102.8 

30 

53.933 

28 

25 

8/28/14 

1332.6 

31 

44.335 

28 

26 

9/2/14 

1041.4 

32 

57.616 

28 

27 

9/3/14 

1177.6 

29 

49.769 

29 

30 

8/19/14 

803.16 

32 

74.705 

29 

30 

8/21/14 

773.91 

32 

77.529 

29 

30 

8/25/14 

666.69 

32 

89.997 

29 

30 

8/28/14 

734.91 

32 

81.643 

29 

30 

9/2/14 

721.81 

31 

79.842 

29 

30 

9/11/14 

824.28 

32 

72.791 

29 

29 

9/15/14 

897.19 

32 

66.876 

29 

30 

9/16/14 

670.16 

32 

89.531 

32 

30 

8/20/14 

787.72 

31 

73.899 

32 

30 

8/28/14 

936.19 

31 

64.449 

32 

30 

9/2/14 

807.75 

32 

74.28 

32 

28 

9/3/14 

813.62 

32 

73.744 

32 

27 

9/4/14 

862.41 

31 

70.394 

33 

29 

8/26/14 

648.34 

31 

88.854 

33 

30 

8/28/14 

683.5 

31 

84.204 

33 

30 

9/3/14 

751.19 

32 

79.874 

33 

29 

9/4/14 

742.66 

32 

80.791 

patientName  MMSE  Score  TestDate  meanRT(CSLl)  NumCorrect(CSLl)  meanThru(CSLl) 


1 

26 

1/13/14 

1846.8 

24.99984 

20.008 

1 

25 

1/14/14 

2130.1 

24.99984 

20.631 

1 

20 

1/23/14 

2759.5 

27 

16.987 

1 

25 

1/27/14 

2806.3 

29.00016 

18.188 

1 

22 

1/28/14 

2406.6 

30.99996 

21.903 

1 

19 

2/3/14 

2480.9 

33.00012 

23.276 

1 

23 

2/4/14 

2928.8 

28.00008 

17.534 

1 

22 

2/10/14 

2817.7 

24.00012 

16.009 

1 

22 

2/11/14 

2283.7 

32.00004 

23.586 

2 

29 

1/22/14 

1415.7 

34.99992 

40.949 

2 

29 

1/28/14 

1227.5 

36 

48.881 

2 

28 

2/18/14 

1165.8 

36 

51.467 

2 

30 

2/24/14 

1292.8 

33.99984 

43.728 

2 

29 

3/5/14 

1258.6 

36 

47.671 

3 

25 

2/4/14 

2219 

32.00004 

25.629 

3 

27 

2/10/14 

1594.6 

33.99984 

35.002 

3 

30 

2/11/14 

1599.3 

33.99984 

36.623 

3 

30 

2/17/14 

1830.8 

32.00004 

29.735 

3 

29 

2/18/14 

2220.8 

29.00016 

24.796 

3 

29 

3/10/14 

1555.6 

34.99992 

39.261 

3 

27 

3/21/14 

1349.7 

36 

44.455 

3 

30 

3/24/14 

1434.2 

36 

41.836 

3 

29 

3/25/14 

1428.1 

36 

42.014 

3 

27 

3/31/14 

1831.3 

33.00012 

32.86 

3 

30 

4/1/14 

2072.6 

33.00012 

27.492 

3 

28 

4/9/14 

1844.4 

34.99992 

32.719 

3 

27 

4/10/14 

2044.6 

34.99992 

29.332 

3 

28 

4/14/14 

2100.3 

36 

28.567 

3 

29 

4/15/14 

1457.6 

36 

41.162 

3 

29 

4/21/14 

2115.9 

36 

28.357 

3 

27 

4/22/14 

1696.8 

34.99992 

34.748 

3 

27 

5/5/14 

2223.8 

33.00012 

25.989 

3 

26 

6/19/14 

1748 

34.99992 

34.646 

3 

29 

7/2/14 

1739.3 

33.00012 

31.958 

3 

28 

7/9/14 

1719.8 

36 

34.888 

3 

28 

7/18/14 

1557.1 

36 

38.532 

3 

29 

7/23/14 

1731.9 

33.00012 

31.808 

5 

28 

2/11/14 

1762.9 

33.99984 

32.254 

5 

30 

2/18/14 

1586.7 

36 

37.814 

5 

30 

2/24/14 

1460.1 

36 

41.094 

5 

30 

2/25/14 

1504.6 

34.99992 

39.24 

6 

30 

2/6/14 

2099.1 

34.99992 

28.527 

6 

30 

2/10/14 

1924.1 

36 

31.184 

6 

29 

2/11/14 

1719.3 

34.99992 

33.708 

10 

30 

3/20/14 

1150.2 

36 

52.165 

10 

30 

3/25/14 

1138.9 

36 

52.683 

10 

30 

3/31/14 

1221.8 

36 

49.108 

10 

30 

4/1/14 

1247.1 

36 

48.11 

10 

29 

4/9/14 

1126.1 

36 

53.281 

10 

30 

4/10/14 

1340.4 

36 

44.761 

meanRT(CSLl) 


MMSE  Score 


- Linear  Forecast  Trendline 


NumCorrect(CSLl) 


MMSE  Score 


- Linear  Forecast  Trendline 


meanThru(CSLl) 


MMSE  Score 


- Linear  Forecast  Trendline 


10 

30 

4/15/14 

1506.1 

34.99992 

39.097 

10 

30 

4/22/14 

1347 

36 

44.543 

10 

30 

4/24/14 

1383.3 

34.99992 

42.282 

14 

27 

7/29/14 

2477.3 

32.00004 

22.381 

19 

29 

6/5/14 

2041.7 

34.99992 

28.612 

19 

30 

6/10/14 

2021 

36 

29.688 

19 

30 

6/17/14 

2566.7 

33.00012 

21.837 

20 

28 

6/4/14 

2130.9 

36 

28.157 

20 

28 

6/11/14 

2172.9 

36 

27.613 

20 

26 

6/13/14 

2591.7 

34.99992 

22.863 

20 

27 

6/16/14 

2201.8 

34.99992 

26.302 

20 

29 

7/9/14 

2112.4 

34.99992 

27.564 

21 

30 

7/28/14 

994.67 

36 

60.322 

21 

30 

8/5/14 

1139.9 

34.99992 

51.525 

21 

30 

8/11/14 

1260.6 

34.99992 

46.161 

21 

30 

8/12/14 

1652.5 

36 

36.308 

21 

30 

8/18/14 

1595.9 

34.99992 

38.196 

21 

30 

8/19/14 

1721.2 

34.99992 

33.635 

21 

30 

8/21/14 

1403.6 

36 

42.748 

21 

30 

8/25/14 

1359.6 

34.99992 

43.277 

21 

30 

8/26/14 

1328.2 

36 

45.174 

21 

29 

8/28/14 

1638 

34.99992 

37.143 

21 

30 

9/4/14 

1679.1 

36 

35.734 

21 

27 

9/9/14 

1791.9 

36 

33.484 

21 

30 

9/11/14 

1619.4 

34.99992 

37.602 

22 

30 

7/29/14 

1512.6 

36 

39.666 

23 

27 

7/9/14 

2608.7 

33.99984 

22.426 

23 

30 

7/24/14 

2121.9 

32.00004 

25 

23 

28 

7/29/14 

2365.8 

32.00004 

23.457 

24 

30 

8/18/14 

2855.8 

30.99996 

18.764 

24 

29 

8/19/14 

2817.8 

25.99992 

17.363 

24 

30 

8/21/14 

3138.7 

27 

15.673 

25 

30 

7/31/14 

1325.7 

34.99992 

44.018 

25 

30 

8/5/14 

1397.7 

36 

42.928 

25 

30 

8/7/14 

1651.5 

36 

36.331 

25 

30 

8/12/14 

1710.2 

34.99992 

33.867 

25 

30 

8/14/14 

2225.5 

32.00004 

25.356 

25 

30 

8/19/14 

1660.8 

33.99984 

34.133 

25 

30 

8/21/14 

2343.9 

32.00004 

23.618 

25 

29 

8/25/14 

1775.7 

34.99992 

33.07 

27 

29 

8/5/14 

2276.9 

33.99984 

25.777 

27 

29 

8/7/14 

2421.2 

33.00012 

24.148 

27 

30 

8/12/14 

1952.4 

34.99992 

30.272 

27 

29 

8/18/14 

2007.6 

36 

29.886 

27 

28 

8/19/14 

1795.1 

36 

33.425 

27 

28 

8/22/14 

2073.1 

36 

28.942 

27 

30 

8/26/14 

1911.7 

36 

31.386 

27 

27 

8/28/14 

2271.4 

33.99984 

26.118 

27 

28 

9/2/14 

2239.5 

33.99984 

26.53 

27 

29 

9/5/14 

1927.2 

34.99992 

30.297 

27 

30 

9/9/14 

1604.1 

36 

37.404 

27 

21 

9/18/14 

2486.2 

32.00004 

23.219 

27 

27 

9/23/14 

2346.1 

32.00004 

24.258 

27 

25 

9/25/14 

2284.9 

33.99984 

25.946 

27 

28 

9/29/14 

2165.6 

33.99984 

26.603 

27 

30 

10/7/14 

2077.1 

33.99984 

27.958 

28 

29 

8/13/14 

3047.3 

28.00008 

16.645 

28 

29 

8/18/14 

3032.9 

25.99992 

15.728 

28 

27 

8/21/14 

3170.1 

25.99992 

15.101 

28 

24 

8/26/14 

2700.4 

29.99988 

20.159 

28 

25 

8/28/14 

2896.1 

24.00012 

15.158 

28 

26 

9/2/14 

2914.8 

29.99988 

18.183 

28 

27 

9/3/14 

2847.8 

28.00008 

17.421 

29 

30 

8/19/14 

1850 

33.99984 

31.061 

29 

30 

8/21/14 

1977.2 

36 

30.346 

29 

30 

8/25/14 

1984.7 

33.00012 

29.58 

29 

30 

8/28/14 

1603.3 

34.99992 

36.258 

29 

30 

9/2/14 

1926.3 

34.99992 

31.244 

29 

30 

9/11/14 

1451.4 

33.99984 

39.506 

29 

29 

9/15/14 

1815.9 

36 

33.041 

29 

30 

9/16/14 

1413.8 

36 

42.439 

32 

30 

8/20/14 

2419.4 

33.99984 

23.779 

32 

30 

8/28/14 

2123.9 

36 

28.25 

32 

30 

9/2/14 

1679.8 

32.00004 

31.548 

32 

28 

9/3/14 

1881.7 

36 

31.886 

32 

27 

9/4/14 

1794.7 

33.99984 

31.279 

33 

29 

8/26/14 

1304.4 

36 

46 

33 

30 

8/28/14 

1438 

36 

41.724 

33 

30 

9/3/14 

1595.9 

34.99992 

37.359 

33 

29 

9/4/14 

1779.1 

33.99984 

34.375 

patientName 


MMSE  Score 

TestDate 

meanRT(SPl)  NumCorrect(SPl) 

meanThru(SPl) 

1 

19 

2/3/14 

2890.2 

15 

15.657 

1 

22 

2/11/14 

1648.5 

14 

22.766 

2 

29 

1/22/14 

1681.2 

20 

35.688 

2 

29 

1/28/14 

1659 

19 

34.759 

2 

28 

2/18/14 

1809.2 

20 

33.163 

2 

30 

2/24/14 

2019.2 

20 

29.715 

2 

29 

3/5/14 

3135.2 

19 

18.644 

3 

25 

2/4/14 

2328.3 

18 

23.661 

3 

27 

2/10/14 

2008 

19 

27.949 

3 

30 

2/11/14 

1604.4 

15 

25.983 

3 

30 

2/17/14 

2230.2 

19 

25.541 

3 

29 

2/18/14 

2496.6 

19 

23.657 

3 

29 

3/10/14 

1718.5 

19 

33.343 

3 

27 

3/21/14 

1767.3 

17 

30.395 

3 

30 

3/24/14 

1593.5 

20 

37.652 

3 

29 

3/25/14 

2201.8 

20 

27.25 

3 

27 

3/31/14 

2792.8 

20 

21.484 

3 

30 

4/1/14 

2608.6 

19 

22.959 

3 

28 

4/9/14 

2166.2 

18 

26.78 

3 

27 

4/10/14 

2707.4 

16 

20.789 

3 

28 

4/14/14 

2225.2 

19 

27.415 

3 

29 

4/15/14 

3321 

16 

15.699 

3 

29 

4/21/14 

2860.4 

16 

18.958 

3 

27 

4/22/14 

2672.7 

17 

21.137 

3 

27 

5/5/14 

3128.2 

16 

17.663 

3 

26 

6/19/14 

3578.4 

15 

14.41 

3 

29 

7/2/14 

3114.3 

17 

17.451 

3 

28 

7/9/14 

2174.8 

18 

27.127 

3 

28 

7/18/14 

2123.6 

19 

27.101 

3 

29 

7/23/14 

2867.2 

16 

18.193 

5 

28 

2/11/14 

2453 

18 

23.188 

5 

30 

2/18/14 

3091.6 

14 

13.756 

5 

30 

2/24/14 

2402.4 

16 

20.313 

6 

30 

2/6/14 

3128.4 

17 

17.381 

6 

30 

2/10/14 

3228.5 

16 

14.966 

6 

29 

2/11/14 

2883.2 

19 

20.564 

10 

30 

3/20/14 

1801.5 

20 

33.307 

10 

30 

3/25/14 

1699.2 

20 

35.312 

10 

30 

3/31/14 

1984 

20 

30.243 

10 

30 

4/1/14 

1975.8 

20 

30.368 

10 

29 

4/9/14 

1781.3 

20 

33.682 

10 

30 

4/10/14 

2476.9 

19 

23.772 

10 

30 

4/15/14 

2318.3 

20 

25.88 

10 

30 

4/22/14 

2621.5 

18 

22.908 

10 

30 

4/24/14 

1966.8 

19 

29.398 

14 

27 

7/29/14 

2531.4 

20 

23.702 

19 

29 

6/5/14 

3031.6 

17 

16.866 

19 

30 

6/10/14 

2742.4 

16 

19.628 

19 

30 

6/17/14 

2559.6 

17 

20.753 

20 

28 

6/4/14 

2147.8 

20 

27.936 

meanRT(SPl) 


MMSE  Score 


- Linear  Forecast  Trendline 


NumCorrect(SPl) 


MMSE  Score 


- Linear  Forecast  Trendline 


meanThru(SPl) 


MMSE  Score 


- Linear  Forecast  Trendline 


20 

28 

6/11/14 

2994.6 

19 

19.73 

20 

26 

6/13/14 

2902.8 

18 

19.798 

20 

27 

6/16/14 

2744.4 

18 

20.651 

20 

29 

7/9/14 

2688.8 

19 

22.204 

21 

30 

7/28/14 

1445.2 

18 

36.462 

21 

30 

8/5/14 

1816.5 

18 

29.663 

21 

30 

8/11/14 

2054.6 

18 

26.623 

21 

30 

8/12/14 

2010.2 

20 

29.849 

21 

30 

8/18/14 

2607.8 

19 

22.966 

21 

30 

8/19/14 

2262.8 

19 

25.961 

21 

30 

8/21/14 

2439 

19 

24.737 

21 

30 

8/25/14 

2770.1 

19 

21.488 

21 

30 

8/26/14 

2531.4 

18 

23.924 

21 

29 

8/28/14 

2094.4 

20 

28.647 

21 

30 

9/4/14 

2828.7 

18 

20.87 

21 

27 

9/9/14 

3151.2 

18 

18.331 

21 

30 

9/11/14 

2152.7 

20 

27.873 

22 

30 

7/29/14 

2012.2 

20 

29.819 

23 

27 

7/9/14 

3445.5 

15 

14.059 

23 

30 

7/24/14 

3184.8 

16 

16.645 

23 

28 

7/29/14 

3406.1 

15 

15.308 

24 

29 

8/19/14 

3807.8 

14 

12.552 

24 

30 

8/21/14 

3194 

16 

16.872 

25 

30 

7/31/14 

1341.8 

18 

40.719 

25 

30 

8/5/14 

1810.1 

20 

33.147 

25 

30 

8/7/14 

1910.5 

18 

27.73 

25 

30 

8/12/14 

1987.2 

19 

28.2 

25 

30 

8/14/14 

1673.3 

20 

35.857 

25 

30 

8/19/14 

2094.8 

20 

28.642 

25 

30 

8/21/14 

1878 

19 

31.045 

25 

29 

8/25/14 

1721.7 

20 

34.85 

27 

29 

8/5/14 

3320.9 

14 

15.304 

27 

29 

8/7/14 

3132.5 

19 

18.786 

27 

30 

8/12/14 

3421.1 

15 

15.011 

27 

29 

8/18/14 

3209.7 

20 

18.693 

27 

28 

8/19/14 

3331.6 

19 

16.906 

27 

28 

8/22/14 

3506.4 

16 

15.321 

27 

30 

8/26/14 

2495.7 

19 

22.646 

27 

27 

8/28/14 

3208.1 

17 

17.636 

27 

28 

9/2/14 

3283.4 

15 

14.909 

27 

29 

9/5/14 

2653.6 

17 

18.712 

27 

30 

9/9/14 

2421.7 

16 

21.245 

27 

27 

9/23/14 

3484.8 

14 

14.635 

27 

30 

10/7/14 

3073.6 

17 

17.118 

28 

29 

8/13/14 

2402.8 

18 

23.613 

28 

29 

8/18/14 

2090.3 

19 

27.447 

28 

27 

8/19/14 

2816.8 

18 

19.362 

28 

27 

8/21/14 

2444.2 

18 

22.607 

28 

24 

8/26/14 

2460.8 

17 

20.996 

28 

25 

8/28/14 

3131.8 

17 

18.2 

28 

26 

9/2/14 

2070.6 

18 

25.991 

28 

27 

9/3/14 

2597.3 

20 

23.1 

29 

30 

8/19/14 

2587.6 

18 

22.719 

29 

30 

8/21/14 

2234.4 

19 

26.472 

29 

30 

8/25/14 

2828.4 

18 

20.151 

29 

30 

8/28/14 

1945.8 

18 

27.629 

29 

30 

9/2/14 

2439.1 

20 

24.599 

29 

30 

9/11/14 

2798.6 

19 

20.823 

29 

29 

9/15/14 

2728 

19 

21.078 

29 

30 

9/16/14 

2221.8 

17 

22.822 

32 

30 

8/28/14 

3120.8 

18 

18.298 

32 

30 

9/2/14 

2858.3 

18 

19.044 

32 

28 

9/3/14 

3326.5 

18 

16.722 

32 

27 

9/4/14 

3166.9 

17 

17.265 

33 

29 

8/26/14 

1935.8 

19 

29.172 

33 

30 

8/28/14 

2219.2 

19 

25.701 

33 

30 

9/3/14 

3141.2 

14 

15.805 

33 

29 

9/4/14 

3001.3 

15 

16.967 

patientName  MMSE  Score  TestDate  meanRT(MS1  NumCorrect(  meanThru(MSl) 


2 

29 

1/22/14 

902.23 

27.9999 

67.976 

2 

29 

1/28/14 

852.3 

30 

70.398 

2 

28 

2/18/14 

818.7 

30 

73.287 

2 

30 

2/24/14 

832 

30 

72.115 

2 

29 

3/5/14 

934.87 

29.0001 

64.578 

3 

25 

2/4/14 

1374.2 

20.0001 

31.502 

3 

27 

2/10/14 

1535.2 

21.9999 

30.332 

3 

30 

2/11/14 

1264.6 

21 

35.529 

3 

30 

2/17/14 

1345.4 

24 

39.319 

3 

29 

2/18/14 

1329.4 

24.9999 

39.7 

3 

29 

3/10/14 

1097.7 

29.0001 

54.377 

3 

27 

3/21/14 

1151.2 

27.9999 

50.776 

3 

30 

3/24/14 

1045.5 

27 

53.717 

3 

29 

3/25/14 

1229.1 

27.9999 

46.377 

3 

27 

3/31/14 

1194.2 

27 

48.882 

3 

30 

4/1/14 

1180.4 

24.9999 

44.753 

3 

28 

4/9/14 

1341.6 

27.9999 

43.256 

3 

27 

4/10/14 

1282.4 

24 

42.353 

3 

28 

4/14/14 

1190.3 

26.0001 

47.644 

3 

29 

4/15/14 

1220.1 

24 

44.785 

3 

29 

4/21/14 

1317.3 

21 

34.805 

3 

27 

4/22/14 

1477.2 

24.9999 

35.808 

3 

27 

5/5/14 

1256.7 

27 

44.943 

3 

29 

7/2/14 

1256.6 

26.0001 

44.434 

3 

28 

7/9/14 

1247.9 

27 

45.605 

3 

28 

7/18/14 

1058.5 

27 

56.613 

3 

29 

7/23/14 

1354.8 

21 

35.086 

5 

28 

2/11/14 

1107.5 

27.9999 

49.612 

5 

30 

2/18/14 

1130.1 

27 

52.252 

5 

30 

2/24/14 

863.77 

30 

69.463 

5 

30 

2/25/14 

1029.9 

24.9999 

51.839 

6 

30 

2/6/14 

1122.7 

27.9999 

52.828 

6 

30 

2/10/14 

1318.4 

26.0001 

42.76 

6 

29 

2/11/14 

1015.2 

30 

59.1 

10 

30 

3/20/14 

957.23 

24.9999 

54.681 

10 

30 

3/25/14 

963.77 

30 

62.256 

10 

30 

3/31/14 

904.67 

29.0001 

66.905 

10 

30 

4/1/14 

1198.2 

26.0001 

47.696 

10 

29 

4/9/14 

904.7 

30 

66.32 

10 

30 

4/10/14 

1062.4 

30 

56.474 

10 

30 

4/15/14 

1265.4 

24.9999 

43.069 

10 

30 

4/22/14 

1085.8 

29.0001 

55.014 

10 

30 

4/24/14 

1093.8 

29.0001 

54.586 

14 

27 

7/29/14 

1183.8 

27 

48.125 

19 

29 

6/5/14 

1081.6 

29.0001 

55.24 

19 

30 

6/10/14 

973.23 

30 

61.65 

meanRT(MSl) 


MMSE  Score 


- Linear  Forecast  Trendline 


NumCorrect(MSl) 


MMSE  Score 


- Linear  Forecast  Trendline 


meanThru(MSl) 


MMSE  Score 


- Linear  Forecast  Trendline 


20 

28 

6/4/14 

1314.3 

24.9999 

41.024 

20 

28 

6/11/14 

1096.2 

29.0001 

54.458 

20 

26 

6/13/14 

1310.7 

26.0001 

41.541 

20 

27 

6/16/14 

1291 

21 

35.074 

20 

29 

7/9/14 

1164 

26.0001 

45.125 

21 

30 

7/28/14 

716.87 

30 

83.698 

21 

30 

8/5/14 

856.43 

27.9999 

65.849 

21 

30 

8/11/14 

826.13 

30 

72.628 

21 

30 

8/12/14 

810.87 

30 

73.995 

21 

30 

8/18/14 

755 

29.0001 

77.662 

21 

30 

8/19/14 

832.17 

30 

72.101 

21 

30 

8/21/14 

1030.2 

29.0001 

58.191 

21 

30 

8/25/14 

825.73 

30 

72.663 

21 

30 

8/26/14 

890.97 

30 

67.343 

21 

29 

8/28/14 

865.9 

29.0001 

70.151 

21 

30 

9/4/14 

927.77 

27.9999 

64.421 

21 

27 

9/9/14 

1112.1 

29.0001 

53.63 

21 

30 

9/11/14 

1131.8 

27.9999 

52.349 

22 

30 

7/29/14 

947.33 

29.0001 

60.576 

23 

30 

7/24/14 

1527.4 

21.9999 

32.459 

23 

28 

7/29/14 

1460.5 

24 

36.21 

24 

30 

8/18/14 

1532.7 

21 

29.372 

24 

29 

8/19/14 

1541.8 

20.0001 

30.308 

24 

30 

8/21/14 

1584.6 

20.0001 

27.556 

25 

30 

7/31/14 

843.53 

29.0001 

68.763 

25 

30 

8/5/14 

1011.3 

30 

59.328 

25 

30 

8/7/14 

1104 

27.9999 

53.844 

25 

30 

8/12/14 

981.8 

29.0001 

61.266 

25 

30 

8/14/14 

899.8 

27.9999 

62.851 

25 

30 

8/19/14 

1177 

27.9999 

50.08 

25 

30 

8/21/14 

1251.8 

26.0001 

45.745 

25 

29 

8/25/14 

1056.7 

27 

56.088 

27 

29 

8/5/14 

1460.5 

21.9999 

33.986 

27 

29 

8/7/14 

1318.1 

27.9999 

43.25 

27 

30 

8/12/14 

1490.2 

24 

35.222 

27 

29 

8/18/14 

1302.4 

27 

42.987 

27 

28 

8/22/14 

1300.8 

24.9999 

40.128 

27 

30 

8/26/14 

1094 

27.9999 

54.034 

27 

27 

8/28/14 

1446.5 

21.9999 

33.121 

27 

28 

9/2/14 

1335.1 

21.9999 

37.136 

27 

29 

9/5/14 

1175.4 

23.0001 

43.915 

27 

30 

9/9/14 

1026.6 

29.0001 

58.405 

27 

27 

9/23/14 

1494.6 

24 

35.082 

27 

25 

9/25/14 

1526.9 

21.9999 

31.874 

27 

28 

9/29/14 

1287.8 

24.9999 

42.393 

27 

30 

10/7/14 

1440.2 

26.0001 

38.403 

28 

29 

8/18/14 

1502.7 

21.9999 

32.015 

28 

24 

8/26/14 

1516.6 

21 

30.454 

28 

26 

9/2/14 

1443.7 

21.9999 

35.442 

29 

30 

8/19/14 

1226.7 

27 

47.338 

29 

30 

8/21/14 

1300.6 

24.9999 

41.803 

29 

30 

8/25/14 

1260.6 

26.0001 

45.34 

29 

30 

8/28/14 

1019 

30 

58.881 

29 

30 

9/2/14 

1171.5 

29.0001 

50.748 

29 

30 

9/11/14 

1116.3 

29.0001 

53.414 

29 

29 

9/15/14 

1100.1 

29.0001 

50.982 

29 

30 

9/16/14 

1263.5 

27 

45.697 

32 

30 

8/20/14 

1400.9 

24.9999 

38.181 

32 

30 

8/28/14 

1158.6 

24.9999 

49.525 

32 

30 

9/2/14 

1278.8 

29.0001 

45.334 

32 

28 

9/3/14 

1039.8 

30 

57.702 

32 

27 

9/4/14 

1223.3 

27 

45.872 

33 

29 

8/26/14 

1070.2 

27 

53.544 

33 

30 

8/28/14 

967.27 

30 

62.03 

33 

30 

9/3/14 

1252.5 

24 

41.7 

33 

29 

9/4/14 

1276.8 

23.0001 

38.096 

Appendix  P:  ECT  Study  Tables 


ECT  (033) 

Control  (Dana0089) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1610.371 

201.005 

989.150 

279.680 

0.00000 

PRT1 

716.143 

56.614 

545.660 

125.080 

0.00031 

RT1 

386.989 

29.238 

282.620 

56.495 

0.00000 

RT2 

422.440 

31.897 

269.180 

68.975 

0.00000 

ECT  (032) 

Control  (69117337) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

t-test  result 

CSL1 

1965.525 

227.219 

1618.250 

41.790 

0.07316 

PRT1 

836.656 

54.642 

746.360 

18.625 

0.05751 

RT1 

485.169 

80.520 

372.290 

2.249 

0.08960 

RT2 

516.505 

75.202 

384.650 

1.301 

0.04518 

ECT  (029) 

Control  (Dana0858) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1734.794 

221.521 

1967.200 

502.360 

0.06424 

PRT1 

771.502 

82.464 

862.560 

269.600 

0.17669 

RT1 

485.434 

99.178 

383.200 

53.751 

0.00000 

RT2 

499.610 

78.314 

320.270 

96.203 

0.00000 

2 


ECT  (028)  Control  (C003) 


Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

t-test  result 

CSL1 

2965.700 

139.678 

NaN 

NaN 

NaN 

PRT1 

1191.136 

92.771 

889.133 

76.641 

0.00000 

RT1 

572.280 

52.184 

565.829 

81.327 

0.84286 

RT2 

607.540 

47.212 

445.650 

0.000 

0.01439 

ECT  (027) 

Control  (Dana0040) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

2126.295 

214.025 

1758.200 

474.300 

0.00027 

PRT1 

1095.155 

91.142 

674.190 

102.300 

0.00000 

RT1 

683.522 

64.589 

460.980 

84.315 

0.00000 

RT2 

676.489 

62.005 

389.880 

44.333 

0.00000 

ECT (025) 

Control  (Dana0002) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1736.573 

353.819 

1197.600 

372.250 

0.00000 

PRT1 

735.200 

78.489 

684.750 

137.530 

0.22374 

RT1 

445.509 

61.483 

298.320 

52.186 

0.00000 

RT2 

486.956 

71.272 

292.730 

37.354 

0.00000 

3 


Test  Name 

ECT (024) 

meanRT  (M)  meanRT  (SD) 

Control  (Dana0137) 
meanRT  (M)  meanRT  (SD) 

z-test  result 

CSL1 

3060.725 

285.128 

1349.100 

439.660 

0.00000 

PRT1 

1134.135 

227.958 

561.380 

75.788 

0.00000 

RT1 

627.500 

20.903 

357.230 

42.111 

0.00000 

RT2 

594.220 

50.137 

295.950 

38.095 

0.00000 

ECT  (023) 

Control  (70248881) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

t-test  result 

CSL1 

2333.100 

206.878 

1712.250 

99.631 

0.01142 

PRT1 

1183.800 

127.263 

712.905 

134.923 

0.00417 

RT1 

596.400 

70.821 

308.185 

22.436 

0.00165 

RT2 

585.208 

16.688 

305.875 

6.541 

0.00000 

ECT  (021) 

Control  (Dana0018) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1315.495 

240.431 

1512.700 

397.420 

0.01140 

PRT1 

656.662 

80.890 

672.940 

139.840 

0.55282 

RT1 

362.009 

36.717 

350.680 

45.572 

0.20493 

RT2 

357.614 

28.389 

378.950 

54.127 

0.04443 

4 


Test  Name 

meanRT  (M) 

ECT  (020) 
meanRT  (SD) 

Control  (Dana0042) 
meanRT  (M)  meanRT  (SD) 

z-test  result 

CSL1 

2366.775 

237.447 

1924.600 

405.260 

0.00001 

PRT1 

1010.726 

125.624 

864.660 

123.120 

0.00000 

RT1 

531.426 

87.717 

381.200 

100.940 

0.00000 

RT2 

510.595 

100.357 

366.770 

41.411 

0.00000 

ECT  (019) 

Control  (Dana0125) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

2200.200 

230.596 

2156.400 

363.240 

0.78745 

PRT1 

845.124 

72.175 

951.810 

188.160 

0.20485 

RT1 

489.208 

48.120 

491.550 

90.749 

0.95398 

RT2 

503.584 

31.834 

546.150 

104.840 

0.36395 

ECT  (010) 

Control  (Dana0006) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1309.376 

146.159 

1471.700 

457.680 

0.14365 

PRT1 

715.435 

50.757 

608.000 

306.660 

0.14860 

RT1 

400.791 

36.477 

273.750 

66.687 

0.00000 

RT2 

415.775 

26.967 

245.750 

35.735 

0.00000 

5 


ECT  (006) 

Control  (Dana0024) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1864.550 

184.223 

1372.500 

366.290 

0.00722 

PRT1 

779.190 

63.073 

614.660 

166.110 

0.04759 

RT1 

389.010 

19.351 

305.180 

78.474 

0.03264 

RT2 

381.280 

25.806 

247.350 

40.572 

0.00000 

ECT (005) 

Control  (Dana0092) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1716.963 

177.738 

2437.200 

522.310 

0.00010 

PRT1 

735.449 

44.278 

1013.400 

226.350 

0.00023 

RT1 

403.280 

50.902 

420.730 

48.884 

0.28421 

RT2 

427.618 

53.860 

394.150 

60.315 

0.09598 

ECT (003) 

Control  (DanaOllO) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1920.007 

302.651 

1887.300 

432.170 

0.55773 

PRT1 

872.792 

147.258 

753.030 

131.930 

0.00000 

RT1 

487.297 

77.091 

383.820 

53.380 

0.00000 

RT2 

544.540 

67.310 

431.050 

59.246 

0.00000 

6 


Test  Name 

ECT  (002) 
meanRT  (M) 

meanRT  (SD) 

Control  (Dana0009) 
meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

1236.627 

127.691 

1292.000 

440.440 

0.55540 

PRT1 

667.795 

43.626 

575.780 

82.710 

0.00000 

RT1 

332.702 

14.435 

347.320 

44.398 

0.12251 

RT2 

337.151 

16.561 

387.050 

140.820 

0.09651 

ECT  (001) 

Control  (Dana0091) 

Test  Name 

meanRT  (M) 

meanRT  (SD) 

meanRT  (M) 

meanRT  (SD) 

z-test  result 

CSL1 

2449.031 

320.750 

1949.600 

471.360 

0.00013 

PRT1 

1014.382 

159.866 

642.530 

109.820 

0.00000 

RT1 

597.178 

68.127 

309.400 

34.900 

0.00000 

RT2 

597.310 

58.634 

296.520 

33.491 

0.00000 
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Learning  Objectives 


the  end  of  the  session: 

The  participant  will  recognize  the  value  of  home -based 
telemental  health  for  rural  veterans  with  PTSD  compared  to 
in-clinic  treatment. 

The  participant  will  be  able  to  discuss  the  value  of  home- 
based  telehealth  for  improving  emotional  and  neurocognitive 
functioning  in  rural  veterans  with  PTSD. 

The  participant  will  be  able  to  report  what  factors  do  not 
improve  due  to  a  brief  intervention  for  rural  veterans  with 
PTSD  using  home-based  telehealth. 


Problem 


4  Rural  veterans,  like  others  living  in  rural  areas,  are  underserved  for 
medical  specialties 

4  The  YA  in  particular  has  had  unacceptable  wait  times  for  veterans 
who  require  evidence-based  treatment  for  PTSD 

4  Providers  at  VA  Clinical  Based  Outpatient  Clinics  (CBOCs)  are 
either  not  trained  in  EBTs  for  PTSD  or  are  too  busy  with  other 
patients  to  be  able  to  devote  weekly  or  twice-weekly  sessions,  as  is 
standard  of  care 

4  Travel  to  clinics  can  be  prohibitive 

Time  to  travel  in  rural  areas 
Cost  to  travel  (island  to  island) 

Physical  and  psychological  limitations 


Problem 


4*  Prior  research  has  shown  that  tele-mental  health  is  equivalent  in 
most  ways  to  in-clinic  mental  health  treatment  for  rural  veterans 

4  This  research  has  been  conducted  from  one  clinic  to  another  remote 
clinic  or  from  one  room  to  another  to  demonstrate  the  effect 

Morland  et  al  (201 1).  Group  cognitive  processing  therapy  delivered  to 
veterans  via  telehealth:  A  pilot  cohort.  Journal  of  Traumatic  Stress,  24(4), 
465-9. 

Morland  et  al  (2010).  Telemedicine  for  anger  management  therapy  in  a  rural 
population  of  combat  veterans  with  posttraumatic  stress  disorder:  a 
randomized  noninferiority  trial.  J  Clinical  Psychiatry:  7 1(7): 855-63. 

Morland  et  al  (2014).  Cognitive  processing  therapy  for  posttraumatic  stress 
disorder  delivered  to  rural  veterans  via  telemental  health:  a  randomized 
noninferiority  clinical  trial.  J  Clinical  Psychiatry;75(5):470-6. 


Problem 


4  However,  many  Veterans  have  difficulty  getting  care  at 
Community  Based  Outpatient  Clinics  (CBOCs). 

4  Scheduling  is  difficult:  matching  at  time  when  the 
Telehealth  room  is  available 

4  Traveling  to  the  VA-CBOC 

4  For  physically  or  psychologically  injured  Veterans 

4  Travel  from  rural  areas  can  be  prohibitive  due  to 
transportation,  time,  cost 

No-show  and  drop-out  rates  are  very  high  (up  to  50%). 


Solution 


4-  Home-based  Telehealth: 

4  Connects  therapists  with  expertise  in  treating  veteran 
PTSD  with  rural  veterans  who  otherwise  have  difficulty 
obtaining  state  of  the  art  care 


Challenges 


Many  rural  veterans: 

4  Did  not  have  high  speed  internet 
4  Did  not  have  a  computer  or  tablet 
4  Or  did  not  feel  comfortable  using  them 
The  VA  did  not  have  a  home-based  program 

4-  When  they  did  launch  home-based  care,  it  was  using 
system  that  was  quirky  (MOVI/ JABBR) 

4  Needed  downloads  prior  to  every  session 

4  Frequent  dropped  calls 


Challenges 


Federal  Gov’t  requires  very  secure  connection  (FIPS  140-2) 

Cannot  email  patient  information 

4-  difficult  to  review  homework 
Does  not  like  to  loan  out  equipment 

Purchasing  IT  equipment  is  very  difficult 

for  providers 
for  patients 

Contracting  with  service  providers  is  difficult  (Verizon) 


Challenges 

•  ^  ^  • 

Confidentiality 

4-  Verbal  (document  in  medical  records) 

4-  Signed  (causes  delay) 

Safety 

4  What  if  they  state  intention  to  harm 

What  if  they  demonstrate  physical  or  psychological  distress 


Program 


4-  $1M  3-year  demonstration  grant  to  the  VA  Office  of 
Rural  Health 

4-  Staff: 

4-  2.0  FTE  Clinical  Psychologists 
4-  1.5  FTE  Psychology  Technicians  (MA) 

4*  40,  4G  enabled,  10-inch  Samsung  Galaxy  Tablets 
4*  Laptop  computers  for  the  clinicians 
Vidyo  videoconferencing  software 
Verizon  contract 


Treatment 


CPTc  -  Cognitive  Processing  Therapy  (cognitive  only) 

4  Referals  from  primary  MH  provider 

4  We  did  not  want  to  do  intakes  and  manage  their  overall  care 

4  Screening  for  appropriateness  prior  to  treatment 

4  (preliminary  SUD  or  other  treatment  if  necessary) 

4  Orientation  session  (tablets,  therapy,  consents, 
assessments) 

12  twice-weekly  sessions 

Referral  back  to  their  primary  MH  provider 

Monthly  follow-up  calls  as  needed 


Progress 


4  Year  1  -  set  up  and  piloting 

4-  Year  2  -  900  patient  encounters 

4  Oahu  (Waianae;  North  Shore) 

4  Maui,  Kauai,  Hawaii,  Molokai,  Guam,  Saipan 
4  Year  3  -  in  progress 

4  Expanding  to  include  any  post-traumatic  symptomology 

4  Anxiety,  Depression,  Anger,  Post-traumatic  Numbing, 
Insomnia,  Somatic  complaints 


Criteria 


4  No  intent  to  harm  within  past  six  months 
4  No  active  or  immediate  potential  for  psychosis 
4  SUD 

Must  guarantee  to  be  sober  during  sessions 
No  more  than  4  drinks/ day,  and  encouraged  to  reduce 
No  major  psychoactive  recreational  substances 
4  Must  have  ability  to  commit  to  schedule 

4  Must  have  isolated  quiet  room  for  sessions 

4  May  require  Personal  Safety  Person  (PSP),  if  risk  is  elevated  or  they 
are  in  a  remote  area 

4  Must  have  emergency  response  team  within  15  minutes  of  session 
location 


Procedure 


4  Receive  referral 

4-  Contact  veteran 

4  Describe  program 
4  Describe  treatment 
4  Obtain  verbal  agreement 
4  Send  out  consent  and  HIPAA  forms 

4  Veteran  signs  and  return  consents  and  Baseline 
Assessments 


Send  out  tablet,  procedure  guide,  and  CPT  manual  to  vet 


Procedure 


Psych  tech  orients  veteran  to  tablet/Vidyo,  assessments 

4-  Using  4G  Verizon  network  or  WiFi  if  4G  not  available 
Psych  tech  schedules  therapy  sessions 

Psych  tech  Vidyo-calls  veteran  five  minutes  prior  to  each 
session  to  insure  connectivity. 

Therapist  ‘joins’  call,  psych  tech  drops  out 

Continues  for  orientation  and  12  CPT  sessions 


Mid  and  Final  Assessments 


Assessment 


$700,000  grant  from  the  Department  of  Defense  as  part  of  a 
grant  to  Anthrotronix,  Inc,  Silver  Springs,  MD 

Assess: 

4  Emotional  Distress  (PCL,  PHQ,  PSQI) 

4  Neurocognitive  Functioning  (DANA,  NB SI-22) 

Familiarity  and  Comfort  with  Technology 
Groups: 

Of  all  those  interested  in  and  enrolled  in  treatment 

(We  also  sought  a  comparison  group  who  were  eligible  but  not 
interested  in  this  approach  to  treatment  and  sought  in-clinic 
treatment;  however,  only  2  preferred  clinic  Tx/HBTx) 


Assessment 


VA IRB  Approved  protocol  established  to  examine  outcome 
Assessments: 

Baseline  (prior  to  treatment  initiation) 

Mid-treatment  (at  3-4  weeks,  independent  of  #sessions) 
Post-treatment  (1-week;  7-10  weeks,  independent  of  #sessions) 
Analysis 

Included  only  those  who  had  completed  all  three  assessments 

Intention-to-treat:  included  all  those  who  initiated  treatment,  even  if 
they  dropped  out  prior  to  completion  (as  long  as  they  were  able  to 
complete  assessments). 


Interim  Analysis 


Emotional  Factors 
Neurocognitive  Functioning 
Drop-outs 
No-shows 

Comfort  levels  with  technology 


Demographics 


4  78.7%  Male  (3  didn’t  indicate) 


4  74.5%  deployed  within  the  past  five  years 


Ethnic  group  identifies  with 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  White 

13 

27.7 

31.0 

31.0 

Black 

3 

6.4 

7.1 

38.1 

Asian  American 

6 

12.8 

14.3 

52.4 

Pacific  Islander 

13 

27.7 

31.0 

83.3 

Mixed 

3 

6.4 

7.1 

90.5 

Other 

4 

8.5 

9.5 

100.0 

Total 

42 

89.4 

100.0 

Missing  System 

5 

10.6 

Total 

47 

100.0 

Demographics 


38.3%  had  a  blast/ explosion  to  the  head 

97.9%  had  received  a  PTSD  Dx  (Dx  upon  study  entry) 

68%  Depression  Dx 

40.4%  Anxiety  Disorder  Dx 

25.5%  Have  tried  to  kill  self  (60%  within  past  4  years) 
70.2%  Have  hit  in  anger 
23.5%  Have  been  hospitalized  for  MH  problem 
50%  of  these  within  past  3  years 
32%  had  more  than  3  concussions 
68.1%  has  chronic  pain 
55.3%  has  sleep  problems 
29.8%  Insomnia  Dx 


Estimated  Marginal  Means  of  MEASURE_1 


Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MEASURE_1 


50.00- 


45.00- 


40.00- 


35.00- 


30.00- 


Estimated  Marginal  Means  of  MEASURE_1 


Estimated  Marginal  Means  of  MEASURE 1 


Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MEASURE_1 


Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MEASURE_1 


Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MEASURE_1 


Estimated  Marginal  Means  of  MEASURE 1 


No  Improvement  Found 


Insomnia  (PSQI) 

Possibly  insomnia  (Mean=12  on  PSQI)  is  slower  to  recover 
Simple  Reaction  Time 

Procedural  Reaction  Time 

SRT  and  PRT  are  more  indicative  of  white  matter  function 

4-  GNG-impulsivity;  CDL-attention;  CDR-STM;  SPD  are  indicative  of 
executive  functioning 

Perhaps  physiological  changes  are  slower  to  occur,  requiring 
continued  treatment  (or  longer  follow-up). 

Other  studies  have  not  examined  neurocognitive  changes  due  to 
treatment  of  PTSD,  so  difficult  to  compare. 


Other  changes 

4-  About  1 8%  drop-outs 

4-  Much  better  than  is  found  in  CBOCs  with  PTSD  patients 
(which  can  run  as  high  as  50%) 

4-  Low  no-show  rate  (hard  to  no-show  when  one  is  at  home) 

4  Saving  on  travel 

4  Veterans  typically  seen  within  two  weeks  of  referral 
4  Very  high  patient  satisfaction  with  Tx  and  technology 
4  Willing  to  see  future  MH  professionals  via  HBTH 
4  Willing  to  see  other  healthcare  professionals  via  HBTH 


How  did  Veterans  feel  about 


10.  Assuming  the  doctor  believes  either  option  is  fine,  which  would  you  prefer? 
a.  For  mental  health  issues: 


MID 

Prefers  this  mode  for  MH  issues 

Total 

Going  to  clinic 

Home-based 

Doesn't  matter 

to  receive  F2F 

televideo  with 

to  me 

with  Dr. 

Dr 

Total 

3 

12 

13 

28 

b.  For  physical  health  follow-up  sessions: 


MID 

Prefers  this  mode  for  physical  health  follow  up 

Total 

sessions 

Going  to  clinic 

Home-based 

Doesn't  matter 

to  receive  F2F 

televideo  with 

to  me 

with  Dr. 

Dr 

Total 

_ 11 

_ 8 

_ 10 

_ 29] 

II  am  comfortable  interacting  with  the  provider  on  the 
computer/webcam/tablet 


MID 

comfortable  interacting  with  the 

Total 

provider  on  the  computer/ 

webcam  /  tablet. 

Agree 

Strongly  Agree 

Total 

17 

13 

30 

'8).  I  feel  I  have  the  same  level  of  rapport  with  my  MH  Provider  when  participating 
lin  HBTMH  as  I  do  in  face-to-face  appointments. 


MID 

same  level  of  rapport  with  myHBMTH  Provider  as  1  do  in  face-to-face 

Total 

appointments. 

Disagree 

Neutral 

Agree 

Strongly  Agree 

2 

_ 3^ 

_ 13, 

12 

_ 30^ 

Next  Steps 


Sustainment  hoped  for 

4-  Expecting  that  VA  will  put  more  emphasis  into  home-based 
telemental  health,  including  sending  out  tablets  when 
necessary 

4-  We  have  showed  that  the  cost  to  deliver  HBTMH  for  veterans 
with  PTSD  is  far  less  than  the  benefits  received 

4*  Monetarily  (VERA  calculations  of  VA  received  to  support  tx) 

4-  Time  to  initial  and  completing  treatment 

Applying  for  grants  to  determine  the  feasibility  and 
effectiveness  of  Behavioral  Health  and  PC  integration  via 
HBTMH 
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1  Overview 


The  data  are  summaries  of  repeated  measures  of  Defense  Automated  Neurobehavioral  Assessments  (DANA) 
on  two  groups  of  fighters.  Each  assessment  is  comprised  of  three  tests:  “go/no-go”  (GNG),  simple  reaction 
time  (SRT),  and  procedural  reaction  time  (PRT).  Within  each  study  group,  all  tests  were  performed  during 
the  course  of  a  single  night.  At  the  beginning  of  the  night,  each  test  was  administered  to  establish  pre¬ 
fight  baseline  measurements.  After  fighting  began,  post-baseline  tests  were  administered  to  one  group  at  a 
single  subsequent  time  point,  and  to  the  other  group  at  three  subsequent  time  points.  The  results  of  each 
assessment  include  mean  and  median  reaction  time  (RT),  mean  and  median  correct,  and  mean  and  median 
thruput. 

There  are  two  research  questions: 

1.  Across  the  two  groups  of  fighters,  are  DANA  measures  similar  at  corresponding  points  in  time?  Inves¬ 
tigators  hypothesize  that  (a)  the  distribution  of  the  baseline  measures  will  be  similar  in  the  two  groups, 
and  that  (b)  similarities  in  these  measures  will  “track  in  parallel”  at  subsequent  time  points  during 
the  night.  Thus,  they  expect  to  see  no  meaningful  differences  in  measures  between  the  two  groups  at 
a  given  point  in  time. 

2.  Within  each  of  the  two  study  groups,  are  DANA  tests  “stable”  during  the  night?  Investigators  hypoth¬ 
esize  that  accumulating  physical  fatigue  during  the  night  will  result  in  decrements  in  performance  on 
DANA  measures  during  the  night.  They  expect  to  see  similar  decrements  in  both  groups  of  fighters. 

The  remainder  of  this  document  is  organized  as  follows.  In  Section  2,  we  provide  tables  and  visual 
displays  of  descriptive  statistics  on  each  measure  and  study  group,  for  each  time  point.  In  Section  3,  we  tests 
for  differences  between  the  two  groups  at  baseline,  and  within  each  of  the  two  groups  over  time.  Finally,  the 
descriptive  statistics  displayed  in  the  figures  of  Section  2  are  given  in  table  format  in  the  Appendix,  along 
with  additional  summary  statistics. 

2  Exploratory  Data  Analysis 

To  explore  part  (a)  of  research  question  1,  Figure  1  shows  the  distribution  of  the  DANA  measures  taken  at 
baseline  (time  0)  for  each  of  the  study  groups.  Visual  inspection  of  these  plots  does  not  reveal  any  striking 
patterns  in  the  differences  between  the  two  study  groups  prior  to  fighting. 

Figure  2  displays  the  distribution  of  DANA  measures  for  group  1  (those  who  fought  in  Atlantic  City)  at 
baseline  and  at  the  time  of  the  post-baseline  measurement.  Mean  RT,  RT  correct,  and  thruput  are  on  the 
y-axes,  and  the  baseline  and  post-baseline  measures  are  indicated  on  the  x-axes  with  0  and  1,  respectively. 
For  each  of  the  three  tests  (SRT,  PRT,  and  GNG),  it  appears  that  mean  RT  and  mean  RT  correct  are  higher 
at  time  1  versus  time  0,  and  that  mean  thruput  is  lower  at  time  1  versus  time  0.  Similarly,  Figure  3  shows 
the  distribution  of  the  DANA  measures  for  group  2  (those  who  fought  in  Long  Island)  at  baseline  and  at 
each  of  the  three  post-baseline  time  points  (indicated  with  1,  2,  and  3  on  the  x-axes).  The  plots  for  this 
group  do  not  show  the  same  clear  pattern  of  decline  in  performance  over  time. 

To  examine  potential  time  effects  more  closely,  Figures  4  and  5  display  the  distributions  of  paired  differ¬ 
ences  for  each  measure  and  study  group  at  each  time  point.  Specifically,  Figure  4  displays  the  distribution 
of  each  assessment  measure  taken  on  group  1  at  time  1,  minus  the  corresponding  measure  taken  at  base- 
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line1.  Figure  5  displays  the  distribution  of  each  assessment  measure  taken  on  group  2  at  time  t  minus  the 
corresponding  measure  taken  at  time  0,  for  t  =  1,  2,  and  3  (indicated  on  the  x-axes).  In  each  of  these  plots, 
a  difference  of  zero  (no  difference  over  time)  is  indicated  with  a  horizontal  red  line.  The  paired  differences 
in  Figure  4  give  strong  visual  support  for  the  hypothesis  that  performance  declines  over  time.  Each  of  the 
medians  (thick,  black  center  lines)  of  RT  and  RT  correct  for  each  of  the  three  tests  he  above  0,  indicating 
longer  reaction  times  during  the  post-baseline  assessment.  Furthermore,  all  the  first  quartiles  (bottoms  of 
the  boxes)  of  RT  and  RT  correct  for  SRT  he  above  0.  Similarly,  most  of  the  paired  differences  in  thruput 
fall  below  0,  indicating  lower  thruput  during  the  post-baseline  assessment.  However,  the  plots  in  Figure  5  do 
not  add  support  for  the  hypothesis  of  decline  in  performance  over  time.  On  the  contrary,  plots  of  the  paired 
differences  in  PRT  and  GNG  indicate  improvement  in  performance  over  time,  and  the  paired  differences  in 
SRT  show  no  visual  difference  in  performance  over  time. 

One  factor  that  may  contribute  to  differences  in  post-baseline  measures  between  the  two  study  groups 
is  a  difference  in  testing  protocol.  In  particular,  if  the  post-baseline  assessments  for  one  group  were  taken 
much  sooner  or  later  after  fighting  than  they  were  for  the  other  group,  it  may  cause  differences  in  the 
results.  Figure  6  displays  the  distribution  of  the  time  in  minutes  between  baseline  assessments  and  each 
subsequent  assessment  by  study  group.  Group  1  had  only  one  post-baseline  assessment  taken  at  an  average 
of  approximately  four  hours  after  baseline  measures  were  established.  Group  2  had  three  post-baseline 
assessments  taken  at  averages  of  19,  42,  and  58  minutes,  respectively,  after  baseline  assessment.  These  are 
substantial  differences  and  careful  consideration  should  be  given  when  performing  statistical  analyses  of  the 
two  study  groups  in  combination. 


1The  maximum  difference  in  RT  for  the  mean  of  PRT  (displayed  in  the  upper  left  plot  of  Figure  4)  falls  outside  of  the 
plotting  region. 
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Figure  1:  Distribution  of  Baseline  DANA  Measures  by  Study  Group 
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Figure  2:  Distribution  of  Group  1  DANA  Measures  Over  Time 
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Figure  3:  Distribution  of  Group  2  DANA  Measures  Over  Time 
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Difference  in  Thruput  Difference  in  RT  Correct  Difference  in  RT 


350 


Mean 


Median 


250 

150 

50 

-50 

-150 

-250 


SRT  PRT  GNG 


350 

250 

150 

50 

-50 

-150 

-250 


SRT  PRT  GNG 


75 

50 

25 

0 

-25 

-50 

-75 


-100 


o 


0 


1 -  - r 


o 


SRT  PRT  GNG 


350 

250 

150 

50 

-50 

-150 

-250 


SRT  PRT  GNG 


350 

250 

150 

50 

-50 

-150 

-250 


SRT  PRT  GNG 


50 


o 


25 

0 


-25 


-50 


o 


o 


-75 


o 


-100  - 


SRT  PRT  GNG 


Figure  4:  Differences  in  Group  1  DANA  Measures  Over  Time 
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Figure  5:  Differences  in  Group  2  DANA  Measures  Over  Time 
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Post-Baseline  Assessment 


□  Group  1 

□  Group  2 


Figure  6:  Distribution  of  the  Time  Between  Baseline  Assessment  and  Each  Subsequent  Assessment  by  Study 
Group 


3  Statistical  Inference 

To  address  the  research  questions  with  more  rigor,  we  formally  test  each  hypothesis  detailed  in  Section  1 
with  a  parametric  and  non-parametric  method. 

3.1  Baseline  Assessments 

We  begin  with  an  examination  of  the  DANA  measures  at  time  0.  Tables  1  and  2  display  the  results  of  two- 
sample  t-tests  and  Wilcoxon  rank  sum  tests,  respectively,  for  differences  in  the  two  study  group  at  baseline, 
for  each  of  the  eighteen  measures.  None  of  the  two-sample  t-tests  are  significant  at  the  a  =  0.05  level, 
and,  while  three  of  the  eighteen  Wilcoxon  rank  sum  tests  have  p-values  less  than  0.05,  none  of  the  tests  are 
significant  at  the  a  =  0.05  level  after  correcting  for  multiple  comparisons.  Thus,  we  conclude  that  there  were 
no  significant  differences  between  DANA  measures  of  the  two  study  groups  prior  to  fighting. 
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Table  1:  Two-Sample  t- Tests  of  Equality  in  the  Baseline  Means  of 
Groups  1  and  2 


DANA 

Measure 

Mean 

Group  1  Group  2 

95% 

MOE 

t 

DF 

P-Val 

Mean  RT 

343.70 

322.80 

38.81 

1.09 

40 

0.2829 

Median  RT 

322.15 

298.39 

34.64 

1.39 

40 

0.1734 

SRT 

Mean  RT  Correct 

337.52 

320.89 

38.27 

0.88 

40 

0.3852 

Med  RT  Correct 

321.48 

298.29 

34.66 

1.35 

40 

0.1839 

Mean  Thruput 

180.57 

191.52 

17.94 

-1.23 

40 

0.2243 

Median  Thruput 

189.10 

205.55 

17.87 

-1.86 

40 

0.0703 

Mean  RT 

611.61 

617.17 

57.27 

-0.20 

40 

0.8456 

Median  RT 

580.74 

588.08 

49.46 

-0.30 

40 

0.7658 

PRT 

Mean  RT  Correct 

610.61 

615.11 

55.36 

-0.16 

40 

0.8703 

Median  RT  Correct 

582.50 

587.61 

49.37 

-0.21 

40 

0.8355 

Mean  Thruput 

97.90 

97.63 

8.69 

0.06 

40 

0.9490 

Med  Thruput 

102.22 

102.24 

8.73 

-0.00 

40 

0.9970 

Mean  RT 

558.17 

521.81 

51.29 

1.44 

38 

0.1593 

Med  RT 

526.50 

490.97 

50.11 

1.44 

38 

0.1593 

GNG 

Mean  RT  Correct 

557.11 

523.23 

51.58 

1.33 

38 

0.1916 

Med  RT  Correct 

526.35 

491.06 

50.36 

1.42 

38 

0.1642 

Mean  Thruput 

107.43 

111.77 

10.58 

-0.83 

38 

0.4112 

Med  Thruput 

113.25 

119.60 

10.39 

-1.24 

38 

0.2231 

MOE  is  the  margin  of  error  for  the  difference  in  means,  t  is  the  test  statistic,  and  DF  is  the  degrees  of 
freedom. 


Table  2:  Wilcoxon  Rank  Sum  Tests  for  Baseline  Differences  Be¬ 
tween  Groups 


DANA 

Measure 

w 

P-Val 

FDR 

P-Val 

Mean  RT 

277.0 

0.1440 

0.4341 

Median  RT 

300.5 

0.0394 

0.2671 

SRT 

Mean  RT  Correct 

265.0 

0.2480 

0.4341 

Median  RT  Correct 

298.5 

0.0445 

0.2671 

Mean  Thruput 

162.0 

0.1586 

0.4341 

Median  Thruput 

130.5 

0.0270 

0.2671 

Mean  RT 

214.0 

0.9204 

1.0000 

Median  RT 

216.0 

0.9597 

1.0000 

PRT 

Mean  RT  Correct 

218.0 

1.0000 

1.0000 

Median  RT  Correct 

217.5 

0.9899 

1.0000 

Mean  Thruput 

212.0 

0.8808 

1.0000 

Median  Thruput 

211.0 

0.8596 

1.0000 

Mean  RT 

241.0 

0.2210 

0.4341 

Median  RT 

243.5 

0.1937 

0.4341 

GNG 

Mean  RT  Correct 

237.0 

0.2653 

0.4341 

Median  RT  Correct 

242.0 

0.2081 

0.4341 

Mean  Thruput 

163.0 

0.3855 

0.5783 

Median  Thruput 

152.5 

0.2449 

0.4341 

W  is  the  test  statistic,  and  ‘FDR’  indicates  false  discovery  rate 
adjustment. 
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3.2  Post-Baseline  Assessments 


Part  (b)  of  research  question  1  asks  whether  the  DANA  measures  on  the  two  study  groups  track  in  parallel  at 
each  time  point  after  baseline  tests  were  administered.  Given  the  magnitude  of  the  differences  in  the  timing 
of  post-baseline  testing  between  the  two  study  groups,  we  caution  against  pooling  post-baseline  measures  of 
the  two  groups. 

Research  question  2  asks  whether  the  accumulation  of  physical  fatigue  during  the  night  results  in  decre¬ 
ments  in  performance  on  DANA  measures.  Given  the  differences  in  the  timing  of  assessments,  we  proceed 
with  separate  analyses  of  post-baseline  measures  for  the  two  study  groups.  For  group  1,  we  perform  paired 
t— tests  and  Wilcoxon  signed  rank  tests  of  the  null  hypotheses  that  there  are  no  differences  in  baseline  and 
post-baseline  performance  against  the  alternative  that  post-baseline  performance  is  worse  than  at  baseline 
(i.e.,  post-baseline  reaction  times  are  longer  and  post-baseline  thruput  is  lower).  The  results  of  these  tests 
confirm  our  conclusions  from  visual  inspection  of  Figure  4.  The  ‘Diff  1  -  O’  column  of  Table  3  gives  the  mean 
paired  difference  (time  1  minus  baseline)  for  each  measure.  The  positive  numbers  in  the  rows  for  mean  and 
median  RT  and  RT  correct  indicate  that  reaction  times  after  fighting  were  longer  than  at  baseline.  The 
negative  numbers  in  the  rows  displaying  mean  and  median  thruput  indicate  that  thruput  was  lower  after 
fighting  than  at  baseline.  The  patterns  in  measures  of  RT,  RT  correct,  and  thruput  are  consistent  across 
the  tests  of  SRT,  PRT  and  GNG.  Most  of  the  results  for  SRT  are  significant  at  the  a  =  0.01  level,  and 
all  are  significant  at  the  a  =  0.05  level.  All  tests  for  post-baseline  differences  in  PRT  are  significant  at  the 
0.05  level.  A  couple  of  the  differences  in  RT  for  GNG  are  significant  at  the  0.05  level,  and  the  differences 
in  thruput  for  GNG  are  not  significant.  Results  of  the  non-parametric  equivalents  of  these  tests  (given  in 
Table  4)  lead  to  the  same  conclusions. 


Table  3:  Paired  t- Tests  for  Differences  Between  Baseline  and  Post- 
Baseline  Performance  in  Group  1 


DANA 

Measure 

Diff 

1  -  0 

95% 

MOE 

t 

DF 

P-Val 

FDR 

P-Val 

Mean  RT 

45.88 

9.58 

2.95 

21 

0.0038 

0.0128 

Median  RT 

34.14 

7.53 

3.08 

21 

0.0029 

0.0128 

SRT 

Mean  RT  Correct 

37.23 

7.57 

2.90 

21 

0.0043 

0.0128 

Median  RT  Correct 

30.68 

6.47 

2.98 

21 

0.0036 

0.0128 

Mean  Thruput 

-21.17 

5.49 

-3.58 

21 

0.0009 

0.0080 

Median  Thruput 

-19.86 

5.30 

-3.69 

21 

0.0007 

0.0080 

Mean  RT 

74.31 

13.74 

2.73 

21 

0.0063 

0.0141 

Median  RT 

44.73 

6.00 

2.35 

21 

0.0143 

0.0257 

PRT 

Mean  RT  Correct 

55.58 

10.38 

2.75 

21 

0.0060 

0.0141 

Median  RT  Correct 

39.25 

4.81 

2.28 

21 

0.0166 

0.0272 

Mean  Thruput 

-8.41 

1.40 

-2.58 

21 

0.0087 

0.0174 

Median  Thruput 

-6.67 

0.74 

-2.21 

21 

0.0192 

0.0288 

Mean  RT 

29.95 

1.24 

1.59 

21 

0.0634 

0.0714 

Median  RT 

30.93 

1.96 

1.97 

21 

0.0311 

0.0399 

GNG 

Mean  RT  Correct 

32.46 

0.14 

1.74 

21 

0.0486 

0.0584 

Median  RT  Correct 

31.48 

2.64 

2.07 

21 

0.0256 

0.0354 

Mean  Thruput 

-3.21 

2.28 

-0.71 

21 

0.2424 

0.2424 

Median  Thruput 

-3.48 

1.95 

-0.81 

21 

0.2129 

0.2255 

11 


Table  4:  Wilcoxon  Signed  Rank  Tests  for  Differences  Between  Base¬ 
line  and  Post-Baseline  Performance  in  Group  1 


DANA 

Measure 

w 

P-Val 

FDR 

P-Val 

Mean  RT 

209.0 

0.0030 

0.0128 

Median  RT 

208.0 

0.0043 

0.0128 

SRT 

Mean  RT  Correct 

209.0 

0.0030 

0.0128 

Median  RT  Correct 

207.0 

0.0037 

0.0128 

Mean  Thruput 

34.0 

0.0008 

0.0075 

Median  Thruput 

33.0 

0.0007 

0.0075 

Mean  RT 

203.0 

0.0057 

0.0143 

Median  RT 

189.0 

0.0212 

0.0382 

PRT 

Mean  RT  Correct 

202.0 

0.0064 

0.0143 

Median  RT  Correct 

188.0 

0.0238 

0.0390 

Mean  Thruput 

62.0 

0.0179 

0.0359 

Median  Thruput 

67.0 

0.0271 

0.0407 

Mean  RT 

170.0 

0.0829 

0.0933 

Median  RT 

179.0 

0.0457 

0.0587 

GNG 

Mean  RT  Correct 

174.0 

0.0644 

0.0773 

Median  RT  Correct 

182.5 

0.0358 

0.0495 

Mean  Thruput 

93.0 

0.1451 

0.1451 

Median  Thruput 

91.0 

0.1308 

0.1385 

For  group  2,  we  use  Friedman  tests  and  repeated  measures  analysis  of  variance  (ANOVA)  to  test  for 
differences  in  DANA  measures  between  any  two  time  points,  and  follow  these  analyses  with  with  pairwise 
paired  t-tests  and  pairwise  Wilcoxon  rank  sum  tests  for  differences  in  DANA  measures  for  each  combination 
of  time  points  (1  vs.  0,  2  vs.  0,  3  vs.  0,  2  vs.  1,  3  vs.  1,  and  3  vs.  2).  The  ANOVA  and  Friedman  tests 
for  differences  between  the  measures  at  any  two  time  points  (shown  in  Tables  5  and  6,  respectively)  confirm 
what  we  saw  in  Figure  5.  Namely,  there  are  no  significant  differences  in  any  of  the  measures  of  SRT  across 
the  four  assessments  on  group  2,  and  there  are  several  significant  differences  in  the  measures  of  PRT  and 
GNG  across  the  four  time  points.  Table  7  displays  the  false  discovery  rate-adjusted  p-values  of  the  paired 
t-tests  and  Wilcoxon  rank  sum  tests  for  each  combination  of  the  time  points  for  PRT  and  GNG.  (Because 
the  ANOVA  and  Friedman  tests  for  SRT  were  not  significant,  no  further  testing  was  required.)  Most  of  the 
p-values  for  the  three  post-baseline  tests  compared  to  baseline  (columns  ‘1  vs  O’,  ‘2  vs  O’,  and  ‘3  vs  O’)  are 
significant  at  the  a  =  0.05  level,  and  none  of  the  post-baseline  measures  are  significantly  different  from  the 
measures  at  any  other  post-baseline  time  point.  That  is,  tests  for  differences  in  the  measures  at  times  2  vs. 
1,  3  vs.  1,  and  3  vs.  2  are  not  significant.  Results  of  the  pairwise  comparisons  for  GNG  are  less  significant 
but  are  in  line  with  the  results  for  PRT. 
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Table  5:  Analysis  of  Variance  for  Differences  in  Performance  Across 
Time  in  Group  2 


DANA 

Measure 

F( DFi,  DF2) 

DFi 

df2 

P-Val 

FDR 

P-Val 

Mean  RT 

1.00 

3 

54 

0.4015 

0.5162 

Median  RT 

0.41 

3 

54 

0.7495 

0.7533 

SRT 

Mean  RT  Correct 

1.19 

3 

54 

0.3223 

0.4576 

Median  RT  Correct 

0.43 

3 

54 

0.7342 

0.7533 

Mean  Thruput 

1.17 

3 

54 

0.3305 

0.4576 

Median  Thruput 

0.40 

3 

54 

0.7533 

0.7533 

Mean  RT 

5.56 

3 

54 

0.0021 

0.0102 

Median  RT 

5.66 

3 

54 

0.0019 

0.0102 

PRT 

Mean  RT  Correct 

5.62 

3 

54 

0.0020 

0.0102 

Median  RT  Correct 

5.50 

3 

54 

0.0023 

0.0102 

Mean  Thruput 

5.18 

3 

54 

0.0032 

0.0107 

Median  Thruput 

4.95 

3 

54 

0.0042 

0.0107 

Mean  RT 

4.62 

3 

48 

0.0064 

0.0144 

Median  RT 

2.85 

3 

48 

0.0470 

0.0845 

GNG 

Mean  RT  Correct 

3.14 

3 

48 

0.0337 

0.0674 

Median  RT  Correct 

0.66 

3 

48 

0.5821 

0.6985 

Mean  Thruput 

5.04 

3 

48 

0.0041 

0.0107 

Median  Thruput 

1.44 

3 

48 

0.2429 

0.3975 

Table  6:  Friedman  Tests  for  Differences  in  Performance  Across 
Time  in  Group  2 


DANA 

Measure 

x2 

DF 

P-Val 

FDR 

P-Val 

Mean  RT 

4.39 

3 

0.2224 

0.2668 

Median  RT 

2.49 

3 

0.4767 

0.5361 

SRT 

Mean  RT  Correct 

4.71 

3 

0.1947 

0.2668 

Median  RT  Correct 

2.31 

3 

0.5115 

0.5361 

Mean  Thruput 

4.71 

3 

0.1947 

0.2668 

Median  Thruput 

2.18 

3 

0.5361 

0.5361 

Mean  RT 

7.61 

3 

0.0548 

0.1096 

Median  RT 

11.83 

3 

0.0080 

0.0651 

PRT 

Mean  RT  Correct 

7.61 

3 

0.0548 

0.1096 

Median  RT  Correct 

11.73 

3 

0.0084 

0.0651 

Mean  Thruput 

9.00 

3 

0.0293 

0.0803 

Median  Thruput 

10.90 

3 

0.0123 

0.0651 

Mean  RT 

10.06 

3 

0.0181 

0.0651 

Median  RT 

5.68 

3 

0.1281 

0.2097 

GNG 

Mean  RT  Correct 

8.86 

3 

0.0312 

0.0803 

Median  RT  Correct 

5.68 

3 

0.1281 

0.2097 

Mean  Thruput 

10.20 

3 

0.0169 

0.0651 

Median  Thruput 

4.55 

3 

0.2076 

0.2668 
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Table  7:  Pairwise  Paired  t-tests  and  Wilcoxon  Signed  Rank  Tests 
for  Differences  in  PRT  and  GNG  for  Group  2 


DANA 

Measure 

Test 

1  vs  0 

FDR- Adjusted  P- Values 

2  vs  0  3  vs  0  2  vs  1  3  vs  1 

3  vs  2 

Mean  RT 

t-test 

0.0077 

0.0077 

0.0056 

0.4324 

0.3209 

0.3209 

Wilcox 

0.0280 

0.1011 

0.0280 

0.9632 

0.3410 

0.1803 

Median  RT 

t-test 

0.0197 

0.0078 

0.0054 

0.2727 

0.3723 

0.5673 

Wilcox 

0.1138 

0.0916 

0.0399 

0.5171 

0.4124 

0.4124 

Mean  RT  Correct 

t-test 

0.0071 

0.0071 

0.0048 

0.4258 

0.2883 

0.2883 

PRT 

Wilcox 

0.0386 

0.0895 

0.0278 

0.9265 

0.2922 

0.1803 

Median  RT  Correct 

t-test 

0.0221 

0.0088 

0.0048 

0.2968 

0.3466 

0.5174 

Wilcox 

0.1440 

0.1044 

0.0399 

0.5540 

0.4534 

0.3837 

Mean  Thruput 

t-test 

0.0064 

0.0066 

0.0039 

0.3366 

0.2121 

0.2976 

Wilcox 

0.0464 

0.0464 

0.0278 

0.9632 

0.2696 

0.2181 

Median  Thruput 

t-test 

0.0180 

0.0079 

0.0056 

0.2674 

0.2674 

0.4759 

Wilcox 

0.1428 

0.0800 

0.0474 

0.6441 

0.4534 

0.3652 

Mean  RT 

t-test 

0.0008 

0.0514 

0.0293 

0.7282 

0.2387 

0.0514 

Wilcox 

0.0101 

0.0610 

0.0610 

0.4845 

0.5791 

0.3370 

Median  RT 

t-test 

0.0584 

0.0584 

0.0584 

0.2693 

0.1150 

0.0584 

Wilcox 

0.1766 

0.1766 

0.1766 

0.8128 

0.4591 

0.4885 

Mean  RT  Correct 

t-test 

0.0022 

0.1487 

0.0726 

0.7581 

0.3106 

0.0726 

GNG 

Wilcox 

0.0101 

0.0697 

0.1011 

0.5168 

0.6441 

0.3652 

Median  RT  Correct 

t-test 

0.1478 

0.3372 

0.3372 

0.4946 

0.3805 

0.3152 

Wilcox 

0.2595 

0.2595 

0.2595 

0.8129 

0.4591 

0.6084 

Mean  Thruput 

t-test 

0.0008 

0.0525 

0.0210 

0.8061 

0.1918 

0.0525 

Wilcox 

0.0039 

0.0330 

0.0464 

0.3160 

0.5477 

0.2851 

Median  Thruput 

t-test 

0.0831 

0.2865 

0.1607 

0.5468 

0.2559 

0.0831 

Wilcox 

0.2142 

0.2404 

0.2142 

0.7819 

0.4534 

0.4262 
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Table  8:  Summary  Statistics  for  Group  1 


DANA 

Measure 

Time 

n 

Min 

Q  i 

Median 

Mean 

SD 

Q  3 

Max 

Mean  RT 

0 

23 

261.60 

300.50 

326.60 

343.70 

63.49 

366.80 

541.00 

1 

22 

281.50 

344.50 

388.50 

392.30 

69.15 

435.30 

513.70 

Median  RT 

0 

23 

256.50 

284.00 

302.00 

322.20 

56.23 

348.20 

493.00 

1 

22 

265.00 

312.60 

361.80 

358.30 

56.77 

395.90 

467.00 

Mean  RT  Correct 

0 

23 

261.60 

295.50 

325.30 

337.50 

62.51 

364.60 

541.00 

SRT 

1 

22 

281.50 

330.20 

382.00 

377.20 

57.47 

411.40 

470.80 

Median  RT  Correct 

0 

23 

256.50 

283.00 

302.00 

321.50 

56.23 

348.00 

493.00 

1 

22 

265.00 

312.60 

353.00 

354.20 

53.10 

393.80 

440.50 

Mean  Thruput 

0 

23 

110.90 

164.60 

183.70 

180.60 

27.97 

199.70 

229.40 

1 

22 

114.70 

138.10 

155.10 

158.00 

29.16 

177.20 

213.10 

Median  Thruput 

0 

23 

121.70 

170.40 

189.60 

189.10 

28.00 

211.30 

233.90 

1 

22 

122.60 

145.70 

165.40 

168.00 

29.93 

190.70 

226.40 

Mean  RT 

0 

23 

471.70 

539.30 

613.80 

611.60 

89.74 

655.80 

782.40 

1 

22 

416.20 

582.00 

714.70 

690.50 

132.41 

771.10 

981.80 

Median  RT 

0 

23 

453.50 

527.00 

584.50 

580.70 

73.00 

627.80 

723.00 

1 

22 

399.00 

558.90 

648.80 

629.10 

100.56 

700.60 

855.50 

Mean  RT  Correct 

0 

23 

469.10 

541.00 

610.30 

610.60 

87.82 

656.10 

787.10 

PRT 

1 

22 

418.90 

583.10 

677.00 

670.70 

109.54 

760.60 

836.40 

Median  RT  Correct 

0 

23 

448.00 

535.80 

584.50 

582.50 

73.21 

634.20 

713.00 

1 

22 

399.00 

558.50 

640.50 

625.50 

97.59 

698.10 

817.50 

Mean  Thruput 

0 

23 

73.85 

89.83 

96.56 

97.90 

14.05 

107.50 

123.90 

1 

22 

62.77 

75.64 

85.68 

88.78 

16.37 

100.90 

134.30 

Median  Thruput 

0 

23 

80.36 

92.69 

102.60 

102.20 

13.39 

110.20 

129.70 

1 

22 

64.22 

83.26 

91.57 

94.90 

16.41 

105.60 

141.00 

Mean  RT 

0 

23 

394.70 

511.10 

533.50 

558.20 

83.06 

608.90 

770.10 

1 

22 

405.10 

510.10 

558.90 

589.60 

120.80 

656.10 

913.30 

Median  RT 

0 

23 

388.00 

491.50 

510.00 

526.50 

73.58 

552.20 

749.00 

1 

22 

396.00 

471.20 

536.00 

559.40 

108.67 

615.20 

840.00 

Mean  RT  Correct 

0 

23 

394.70 

511.10 

533.50 

557.10 

85.35 

608.90 

770.10 

GNG 

1 

22 

405.10 

510.10 

558.90 

591.00 

121.61 

665.50 

913.30 

Median  RT  Correct 

0 

23 

388.00 

491.50 

510.00 

526.30 

74.38 

556.00 

749.00 

1 

22 

396.00 

471.20 

536.00 

559.80 

108.86 

621.20 

840.00 

Mean  Thruput 

0 

23 

77.92 

95.84 

111.90 

107.40 

16.50 

116.60 

152.00 

1 

22 

65.69 

88.89 

105.60 

103.90 

20.07 

117.70 

148.10 

Median  Thruput 

0 

23 

80.11 

101.70 

116.70 

113.20 

15.50 

121.30 

154.60 

1 

22 

71.43 

93.94 

110.60 

109.30 

19.78 

124.60 

151.50 
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Table  9:  Summary  Statistics  for  Group  2 


DANA 

Measure 

Time 

n 

Min 

Q  i 

Median 

Mean 

SD 

Q  3 

Max 

0 

19 

251.50 

280.80 

309.10 

322.80 

60.00 

340.70 

486.70 

Mean  RT 

1 

19 

235.60 

265.50 

294.20 

304.80 

50.42 

330.70 

428.80 

2 

19 

268.20 

283.40 

295.80 

338.20 

102.46 

328.50 

620.80 

3 

19 

244.70 

286.40 

309.00 

323.20 

67.13 

339.60 

564.00 

0 

19 

243.50 

266.80 

282.50 

298.40 

54.10 

308.50 

459.00 

Median  RT 

1 

19 

237.00 

254.50 

278.50 

286.40 

36.34 

319.20 

349.50 

2 

19 

240.50 

266.50 

281.00 

297.50 

54.71 

322.80 

470.00 

3 

19 

234.50 

276.20 

293.00 

295.50 

29.82 

311.50 

364.00 

0 

19 

251.50 

280.80 

309.10 

320.90 

59.28 

335.30 

486.70 

Mean  RT  Correct 

1 

19 

239.10 

268.10 

294.20 

301.10 

41.15 

330.70 

393.90 

2 

19 

268.20 

283.50 

302.90 

333.20 

86.36 

328.50 

560.10 

SRT 

3 

19 

244.70 

286.40 

309.00 

321.70 

60.17 

339.60 

527.20 

0 

19 

243.50 

266.20 

282.50 

298.30 

54.17 

308.50 

459.00 

Median  RT  Correct 

1 

19 

237.00 

255.00 

278.50 

286.20 

35.71 

319.20 

348.00 

2 

19 

241.00 

266.50 

281.00 

295.90 

49.72 

322.80 

441.50 

3 

19 

234.50 

276.20 

293.00 

295.30 

29.22 

311.50 

359.00 

0 

19 

123.30 

179.00 

194.10 

191.50 

29.41 

213.70 

238.60 

Mean  Thruput 

1 

19 

152.30 

181.40 

203.90 

201.30 

26.10 

221.00 

244.70 

2 

19 

104.00 

182.60 

197.00 

187.50 

35.61 

210.30 

223.80 

3 

19 

111.00 

176.70 

194.10 

191.00 

29.15 

209.50 

245.20 

0 

19 

130.70 

194.60 

212.40 

205.50 

29.15 

222.90 

246.40 

Median  Thruput 

1 

19 

165.90 

188.00 

215.40 

211.30 

25.50 

233.00 

246.80 

2 

19 

129.10 

185.90 

210.40 

206.30 

30.68 

225.10 

247.90 

3 

19 

163.00 

191.10 

204.80 

204.60 

21.01 

217.20 

255.90 

0 

19 

508.70 

538.50 

604.90 

617.20 

93.40 

689.40 

802.90 

Mean  RT 

1 

19 

470.90 

522.10 

568.30 

568.40 

65.18 

620.90 

710.80 

2 

19 

430.30 

502.10 

561.60 

565.90 

70.84 

617.40 

687.00 

3 

19 

404.30 

514.50 

553.50 

555.20 

76.02 

594.10 

716.30 

0 

19 

488.50 

513.50 

569.50 

588.10 

85.64 

654.50 

738.50 

Median  RT 

1 

19 

459.00 

493.00 

517.00 

544.10 

61.25 

593.80 

656.50 

2 

19 

426.00 

487.20 

537.00 

533.40 

57.51 

584.20 

632.00 

3 

19 

390.00 

498.00 

537.50 

535.70 

68.28 

555.80 

672.00 

0 

19 

510.60 

538.90 

603.30 

615.10 

89.00 

691.60 

790.60 

Mean  RT  Correct 

1 

19 

477.00 

522.70 

551.90 

568.00 

64.47 

620.90 

710.80 

2 

19 

430.30 

502.10 

563.30 

565.20 

70.34 

613.20 

689.70 

PRT 

3 

19 

404.80 

515.70 

551.20 

553.20 

72.21 

590.20 

685.80 

0 

19 

488.50 

511.80 

569.50 

587.60 

85.12 

656.00 

734.00 

Median  RT  Correct 

1 

19 

467.00 

490.20 

517.00 

544.40 

60.81 

593.80 

656.50 

2 

19 

426.00 

490.00 

537.00 

534.30 

57.69 

580.00 

632.00 

3 

19 

389.00 

495.00 

537.50 

534.90 

68.10 

552.00 

670.00 

0 

19 

73.53 

85.39 

97.71 

97.63 

13.63 

108.80 

116.30 

Mean  Thruput 

1 

19 

84.41 

96.65 

104.90 

105.10 

11.46 

113.30 

122.20 

2 

19 

84.27 

96.32 

103.20 

106.50 

14.81 

119.50 

139.40 

3 

19 

79.29 

99.75 

108.00 

108.70 

15.26 

114.50 

143.60 

0 

19 

79.19 

90.03 

105.40 

102.20 

14.56 

114.50 

122.80 

Median  Thruput 

1 

19 

88.93 

99.72 

114.70 

109.60 

11.57 

118.80 

124.20 

2 

19 

93.60 

102.10 

110.90 

112.20 

13.27 

122.40 

140.80 

3 

19 

81.16 

104.80 

111.60 

112.30 

15.33 

119.40 

149.40 

continued  on  next  page. . . 
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Table  9:  Summary  Statistics  for  Group  2  (continued  from  previous  page) 


DANA 

Measure 

Time 

n 

Min 

Q  i 

Median 

Mean 

SD 

Q  3 

Max 

0 

17 

370.60 

466.00 

541.90 

521.80 

73.57 

570.70 

654.50 

Mean  RT 

1 

18 

356.30 

442.70 

490.10 

489.30 

70.61 

528.00 

637.40 

2 

19 

417.90 

458.40 

495.70 

506.60 

63.80 

542.90 

676.50 

3 

19 

395.30 

442.30 

480.20 

480.30 

53.84 

495.90 

599.00 

0 

17 

309.50 

435.00 

465.00 

491.00 

82.33 

557.00 

626.00 

Median  RT 

1 

18 

314.50 

432.50 

483.00 

479.00 

73.09 

526.80 

630.00 

2 

19 

366.50 

438.80 

465.00 

480.50 

61.83 

520.00 

626.00 

3 

19 

397.00 

425.50 

460.00 

462.30 

45.81 

487.00 

579.00 

0 

17 

389.10 

466.00 

541.90 

523.20 

71.09 

570.70 

654.50 

Mean  RT  Correct 

1 

18 

404.30 

442.70 

490.10 

493.00 

65.76 

529.60 

637.40 

2 

19 

417.90 

458.40 

509.10 

511.90 

65.94 

556.60 

676.50 

GNG 

3 

19 

395.30 

442.30 

482.10 

486.10 

57.30 

510.40 

599.00 

0 

17 

310.00 

435.00 

465.00 

491.10 

82.23 

557.00 

626.00 

Median  RT  Correct 

1 

18 

326.00 

432.50 

483.00 

480.80 

71.96 

526.80 

630.00 

2 

19 

416.00 

442.00 

483.00 

488.20 

55.62 

520.00 

626.00 

3 

19 

397.00 

439.00 

464.00 

473.30 

55.91 

493.50 

618.00 

0 

17 

77.09 

103.00 

110.60 

111.80 

16.11 

124.90 

136.90 

Mean  Thruput 

1 

18 

72.83 

109.40 

120.50 

119.30 

19.35 

135.10 

148.40 

2 

19 

47.96 

106.00 

117.90 

115.50 

21.73 

130.00 

143.60 

3 

19 

52.69 

116.80 

124.40 

122.00 

21.67 

135.70 

151.80 

0 

17 

95.85 

104.40 

119.80 

119.60 

16.76 

136.00 

143.20 

Median  Thruput 

1 

18 

92.02 

112.00 

121.20 

122.10 

17.20 

137.70 

149.60 

2 

19 

54.96 

112.50 

124.20 

120.40 

20.89 

133.80 

144.20 

3 

19 

48.54 

121.60 

128.20 

125.50 

22.00 

136.80 

151.10 

IT 


Resnick,  Chodorow,  and  Associates,  LLC 


Appendix  S 


MEMORANDUM 

To:  Cori  Lathan,  Lawrence  Wolpert 

From:  Helaine  Resnick 

Re:  Air  Force  Data  Distributions  and  Critical  Values 

Date:  September  3,  201 3 


This  memo  summarizes  my  impressions  concerning  the  distributions  of  the  “Air  Force” 
DANA  data  and  how  these  distributions  may  relate  to  future  efforts  aimed  at  identifying 
threshold  values  that  can  be  used  for  screening  and  triage. 

The  Air  Force  data  set  contained  information  for  SRT,  PRT  and  GNG  testing  that  was 
conducted  at  three  points  in  time  over  the  course  of  approximately  8  months.  We  examined 
each  of  the  three  tests  at  each  of  the  three  points  in  time.  We  also  examined  both  means 
and  median  values  for  each  test  as  well  as  both  throughput  and  “pre-throughput”  values. 

The  attached  PDFs  files  are  sorted  according  to  each  test,  with  one  PDF  for  each  test. 
These  files  show  smoothed  curves  superimposed  over  histograms  of  the  raw  data  as  I 
received  them.  Each  panel  also  contains  descriptive  statistics,  including  the  sample  size  on 
which  the  curve  is  based,  as  well  as  quartiles,  means  and  standard  deviations. 

Using  the  mean  and  standard  deviation  from  each  curve,  we  calculated  critical  values  at  the 
first,  second  and  third  standard  deviations  above  the  mean  for  each  distribution.  These 
values  can  provide  an  initial  glimpse  about  how  these  data  (when  examined  together  or 
pooled  with  similar  information  from  other  DANA  protocols)  could  be  used  in  the  future  to 
define  thresholds  above  which  an  individual’s  cognitive  function  could  be  defined  as  “out  of 
normal  range.” 

Over  the  course  of  the  Air  Force  data  study,  the  sample  size  diminished  as  time  passed. 

The  initial  data  collection  included  approximately  142  individuals,  of  whom  96  were 
examined  at  the  second  point  in  time,  and  only  approximately  58  providing  data  at  the  third 
time  point.  Across  all  tests  (e.g.  SRT,  PRT  and  GNG)  the  impact  of  diminishing  sample  size 
on  the  shape  of  the  curves  is  evident.  The  curves  are  wider  and  smoother  when  the  sample 
sizes  are  larger  and  tall  and  choppy  as  sample  size  diminishes. 

In  general,  it  does  not  appear  that  the  shapes  of  the  curves  are  influenced  to  a  high  degree 
by  whether  mean  or  median  values  are  used.  For  the  “pre-throughput”  data  (blue  shaded 
data  on  the  spreadsheet),  mean  values  are  slightly  higher  than  median  values,  but  the 
shapes  of  the  curves  are  similar.  Since  threshold  values  that  are  used  for  clinical  purposes 
depend  on  means  and  standard  deviations  from  the  raw  data,  the  Air  Force  data  suggest 
that  if  the  “pre-throughput”  data  are  used  for  this  purpose,  mean  values  will  be  slightly  less 
likely  to  define  someone  as  “out  of  range”  because  these  values  are  slightly  higher  than 
medians.  The  throughput  data  appear  to  show  the  opposite  pattern:  Median  values  are 
higher  than  mean  values.  Thus,  for  the  Air  Force  data,  if  clinical  thresholds  are  based  on 
throughput  values,  use  of  medians  would  likely  result  in  fewer  people  being  tagged  as  out  of 
range  because  the  medians  are  higher  than  the  means. 

1324  Midwood  Place 
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Tel:  202.329.8616 
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It  is  important  to  stress  that  the  range  of  values  for  the  pre-throughput  vs.  throughput  data 
are  quite  different.  The  range  of  throughput  values  is  substantially  smaller  than  the  pre¬ 
throughput  values.  Importantly,  the  shape  of  the  curves  that  are  derived  using  throughput 
data  are  more  symmetric  than  those  for  pre-throughout  values.  That  is,  the  throughput  data 
tend  to  generate  curves  whose  upper  and  lower  tails  are  more  pronounced,  while  the  mean 
values  tend  to  generate  curves  with  very  pronounced  upper  tails  and  a  blunted  or  absent 
lower  tail.  It  is  possible  that  the  symmetric  nature  of  the  throughput  data  may  make  them 
more  favorable  for  use  in  defining  threshold  values  because  the  means  of  the  throughput 
data  will  be  better  centered  in  the  curves. 

The  attached  spreadsheet  contains  a  summary  of  the  sample  sizes  for  each  test  and  time 
point,  as  well  as  means,  standard  deviations  and  critical  values  at  the  first,  second  and  third 
standard  deviation  above  the  mean  for  the  various  types  of  data  that  were  examined.  As 
noted  above,  the  pre-throughput  data  are  in  blue,  and  the  throughput  data  are  in  gray. 
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SRT :  Mean  RT 


Number 

142 

Median 

314.275 

1st  quartile 

285.82 

3rd  quartile 

339.57 

Minimum 

230.07 

Maximum 

554.24 

Mean 

320.9536 

SD 

55.39 

Number 

96 

Median 
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1st  quartile 

280.915 

3rd  quartile 

330.51 

Minimum 

241.38 
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Mean 
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SD 

41.07 

Number 

58 

Median 
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1st  quartile 
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SRT:  Median  RT 


Number 

142 

Median 

297.5 

1st  quartile 

270.5 

3rd  quartile 

327 

Minimum 

219 

Maximum 

516 

Mean 

304.2887 

SD 

50.38 

Number 

96 

Median 

288.25 

1st  quartile 

270.75 

3rd  quartile 

312.5 

Minimum 

233.5 

Maximum 

425 

Mean 
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SD 
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58 

Median 
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SRT:  Mean  TPT 


Number 

142 

Median 

190.735 

1st  quartile 

175.77 

3rd  quartile 

209.92 

Minimum 

100.14 

Maximum 

260.78 

Mean 

190.8799 

SD 

29.67 

Number 

96 

Median 

197.17 

1st  quartile 

179.845 

3rd  quartile 

213.59 

Minimum 

133.67 

Maximum 

248.58 

Mean 

195.9768 

SD 

24.76 

Number 

58 

Median 
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SRT:  Median  TPT 


Number 

142 

Median 

201.34 

1st  quartile 

183.21 

3rd  quartile 

221.81 

Minimum 

107.56 

Maximum 

273.97 

Mean 

200.9323 

SD 

30.12 

Number 

96 

Median 

207.61 

1st  quartile 

191.12 

3rd  quartile 

220.59 

Minimum 

141.18 

Maximum 

256.96 

Mean 

207.4245 

SD 

23.78 

Number 

58 

Median 

221.605 

1st  quartile 

199.34 

3rd  quartile 

233.07 

Minimum 

163.41 

Maximum 
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Mean 
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SD 
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PRT :  Mean  RT 
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Number 

142 

Median 

580.195 

1st  quartile 

532.03 

3rd  quartile 

634.38 

Minimum 

417.42 

Maximum 

946.28 

Mean 

589.9987 

SD 

88.16 

Number 

96 

Median 

561.105 

1st  quartile 

509.245 

3rd  quartile 

606.64 

Minimum 
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Maximum 
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Mean 
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SD 
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Number 
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Median 
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PRT:  Median  RT 


Number 

142 

Median 

546.5 

1st  quartile 

510.5 

3rd  quartile 

604 

Minimum 

387 

Maximum 

927.5 

Mean 

560.6972 

SD 

84.30 

Number 

96 

Median 

531.5 

1st  quartile 

487.5 

3rd  quartile 

582.5 

Minimum 

403 

Maximum 

779.5 

Mean 

538 

SD 

65.31 

Number 

58 

Median 
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Mean 
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SD 
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PRT :  Mean  TPT 


Number 

142 

Median 

100.735 

1st  quartile 

93.157 

3rd  quartile 

111.01 

Minimum 

63.406 

Maximum 

139.25 

Mean 

101.7845 

SD 

13.68 

Number 

96 

Median 

106.055 

1st  quartile 
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3rd  quartile 
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Minimum 
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Mean 
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SD 
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PRT:  Median  TPT 


Number 

142 

Median 

106.81 

1st  quartile 

98.522 

3rd  quartile 

115.79 

Minimum 

64.69 

Maximum 

150.19 

Mean 

107.0708 

SD 

14.19 

Number 

96 

Median 

111.385 

1st  quartile 
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3rd  quartile 

122.14 

Minimum 

76.972 

Maximum 
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Mean 
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SD 

12.59 

Number 

58 

Median 
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GNG:  Mean  RT 


Number 

142 

Median 

494.965 

1st  quartile 

443.53 

3rd  quartile 

550 

Minimum 

377.8 

Maximum 

929.33 

Mean 

507.1318 

SD 

87.06 

Number 

96 

Median 
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Minimum 
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Maximum 
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Mean 
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SD 
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Number 
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GNG:  Median  RT 


Number 
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Median 
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Mean 
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430.5 

3rd  quartile 

515 

Minimum 

373 

Maximum 

753 
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GNG:  Mean  TPT 


Number 
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GNG:  Median  TPT 
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1  Overview 


The  data  are  summaries  of  paired  repeated  measures  of  Defense  Automated  Neurobehavioral  Assessments 
(DANA)  and  Automated  Neuropsychological  Assessment  Metrics  (ANAM).  Each  assessment  is  comprised  of 
four  tests:  code  substitution  delayed  (CSD),  code  substitution  learning  (CSL),  simple  reaction  time  (SRT), 
and  procedural  reaction  time  (PRT).  The  results  of  each  assessment  include  mean  reaction  time  (RT)  and 
mean  thruput.  The  objective  of  this  analysis  is  to  understand  how  DANA  measures  differ  from  ANAM 
measures  on  similar  tests.  The  two  testing  schemes  use  different  platforms/devices  and  this  is  thought  to 
result  in  a  systematic  measurement  difference  in  RT  of  about  50  milliseconds  between  the  two  devices.  This 
analysis  explores  differences  in  thruput  as  measured  by  the  two  devices,  and  investigates  the  hypothesis  that 
there  is  a  fifty  millisecond  difference  between  DANA- measured  and  AN  AM-measured  reaction  times. 

2  Data 

We  begin  with  a  description  of  the  data  used  in  this  analysis.  Table  1  lists  the  names  of  each  of  the  six  data 
sets  along  with  the  provided  description  of  each  data  source. 

Table  1:  Description  of  Data  Sources 


File  Name 

Provided  Description 

DANAvANAM_DANA_Controls.csv 

SRT,  PRT,  CSD  and  CSL  measured  by  DANA  on  17  non-injured 
subjects 

DANAvANAM_ANAM_Controls.csv 

SRT,  PRT,  CSD  and  CSL  measured  by  ANAM  on  the  same  17 
non-injured  subjects  in  the  DANA  controls  file 

DAN  A  v  AN  AM  _F  lagged_D  AN  A .  csv 

Information  on  DANA  administrations  that  did  not  pass  quality 
control  standards  (to  be  excluded  from  analysis) 

ANAM_data(2)  (2).xls 

Subject  IDs  and  injury  dates  for  injured  individuals  (should  not  be 
found  in  controls  data) 

Email  from  Dr.  Resnick  on  9/10/14 

List  of  subjects  with  concussions  (should  not  be  found  in  controls 
data) 

AN  AM  _D  AN  A  _sub  j  ect  s  _key.  csv 

Linking  file  to  merge  injury  data  with  controls  data 

Upon  inspection  of  the  various  data  sources,  we  found: 

•  twenty-three  subjects  in  the  ANAM  controls, 

•  twenty- two  subjects  in  the  DANA  controls, 

•  two  subjects  in  the  injuries  file  (ANAM_data(2)  (2).xls)  who  are  not  in  the  key  file 
(ANAM_DANA_subjects_key.csv)  and  therefore  can  not  be  linked  to  the  controls  data, 

•  two  subjects  in  the  list  of  individuals  with  concussions  who  are  not  in  the  injuries  file,  and 

•  one  subject  with  ANAM  test  dates  in  2013  and  DANA  test  dates  in  1970. 

After  excluding  individuals  found  in  either  the  injuries  file  or  the  concussion  list,  we  are  left  with  eighteen 
non-injured  subjects  who  have  at  least  one  valid  set  of  both  DANA  and  ANAM  tests.  For  a  count  of  subjects 
by  test  and  assessment  number,  please  see  Tables  3  and  4  in  the  appendix. 
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3  Results 


Figure  1  displays  the  paired  differences  between  mean  DANA- measured  and  ANAM-measured  reaction  time 
and  thruput  by  test  and  time  (assessment  number).  In  each  of  the  eight  sub-figures,  time  is  shown  on  the 
x-axis  and  the  paired  difference  (DANA  -  ANAM)  in  milliseconds  is  shown  on  the  y-axis.  In  the  plot  of 
paired  differences  for  SRT  (upper  left  sub-figure),  the  distributions  of  the  difference  in  mean  reaction  times 
are  roughly  symmetric  around  fifty  milliseconds  for  assessments  1,  2,  3,  4,  and  6,  and  roughly  symmetric 
around  seventy- five  for  assessment  5.  The  plot  of  differences  in  mean  reaction  times  in  tests  of  PRT  (column 
1,  row  2)  also  reveals  distributions  that  are  roughly  symmetric  around  fifty  milliseconds  for  each  of  the 
three  paired  assessments.  Tests  of  CSD  (column  1,  row  3)  show  similar  differences  in  mean  reaction  times 
in  assessments  1  and  3,  and  greater  variability  in  the  paired  differences  from  assessment  2.  Tests  of  CSL 
(bottom  left  sub-figure)  show  differences  in  reaction  times  that  are  much  greater  than  fifty  milliseconds  - 
almost  300  milliseconds  in  the  first  assessment  and  approximately  150  milliseconds  in  the  second  and  third 
assessments. 

Plots  of  the  paired  differences  in  mean  thruput  (column  2  of  Figure  1)  show  analogous  patterns  to  those 
of  mean  reaction  times.  Specifically,  the  paired  differences  in  mean  thruput  during  tests  of  SRT  are  roughly 
symmetric  about  the  same  number  -  negative  forty  -  for  assessments  1,  2,  3,  4,  and  6,  and  the  differences 
for  the  fifth  assessment  are  of  slightly  greater  magnitude.  Differences  in  mean  thruput  in  tests  of  PRT 
are  roughly  symmetric  about  negative  ten  for  all  three  assessments.  Assessments  1  and  3  of  CSD  show 
similar  distributions  of  paired  differences  in  mean  thruput  and  assessment  2  shows  greater  variability  in  the 
differences.  Finally,  the  tests  of  CSL  show  significant  differences  in  DANA-measured  and  ANAM-measured 
thruput  that  vary  in  magnitude  across  the  three  assessments. 

Given  the  relatively  small  sample,  we  formally  test  for  differences  between  DANA  and  ANAM  measures 
with  the  Wilcoxon  signed  rank  test.  Table  2  displays  the  Wilcoxon  statistic  ( W )  and  p- value  for  tests  of  the 
hypothesis  that  DANA-measured  reaction  times  are  greater  than  ANAM-measured  reaction  times  by  fifty 
milliseconds  in  tests  of  SRT,  PRT,  and  CSD.  We  did  not  test  this  hypothesis  for  CSL  as  it  was  made  clear 
in  Figure  1  that  the  differences  for  this  test  are  much  greater  than  fifty  milliseconds.  With  the  exception  of 
the  fifth  assessment  of  SRT,  we  do  not  reject  the  hypothesis  of  a  fifty  millisecond  difference  in  mean  reaction 
time. 
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Figure  1:  Paired  Differences  Between  DANA  and  ANAM  by  Test  and  Time 
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Table  2:  Wilcoxon  Signed  Rank  Tests  for  Differences  of  Fifty 
Milliseconds  in  DANA-Measured  and  AN  AM-Measured  Reaction 
Time  During  Tests  of  SRT,  PRT,  and  CSD 


Test 

t 

w 

P-Val 

1 

99 

0.5798 

2 

106 

0.3927 

3 

72 

0.8536 

SRT 

4 

87 

0.6441 

5 

117 

0.0003 

6 

75 

0.4212 

1 

111 

0.2837 

PRT 

2 

67 

0.6777 

3 

53 

0.7197 

1 

103 

0.4683 

CSD 

2 

44 

0.1324 

3 

64 

0.8469 
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A  Supplemental  Information 


This  section  provides  supplemental  information  compiled  during  analysis.  Figures  2  and  3  display  the 
distributions  of  mean  reaction  time  and  mean  thruput,  respectively,  by  test,  time  and  device.  The  overall 
median  (ignoring  assessment  number)  for  each  test,  measure  and  device  is  shown  as  a  horizontal  blue  line  in 
each  sub-figure.  The  statistics  displayed  in  these  figures  along  with  additional  summary  statistics  are  given 
in  Tables  3  and  4.  To  investigate  patterns  over  time,  Figures  4  and  5  display  paired  differences  in  mean 
reaction  time  and  mean  thruput,  respectively,  within  device.  In  these  figures,  assessments  2,  3,  4,  5,  and 
6  are  compared  with  the  first.  Finally,  Table  5  displays  the  Wilcoxon  signed  rank  statistic  and  p- value  for 
tests  of  the  hypothesis  that  DANA-measured  thruput  minus  AN  AM-measured  thruput  is  equal  to  /iq  (given 
in  column  2  of  the  table)  in  tests  of  SRT  and  PRT.  As  in  the  results  of  tests  for  differences  in  reaction  time, 
the  tested  hypothesis  is  only  rejected  in  the  fifth  assessment  of  SRT. 


6 


CSL  CSD  PRT  SRT 


DANA 


ANAM 


1900 

1700 

1500 

1300 

1100 

900 

700 


1  2  3 


1900 

1700 

1500 

1300 

1100 

900 

700 


1  2  3 


1900 

1700 

1500 

1300 

1100 

900 

700 


1  2  3 


Figure  2:  Distribution  of  Mean  Reaction  Time  by  Test,  Time,  and  Device 
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Figure  3:  Distribution  of  Mean  Thruput  by  Test,  Time,  and  Device 
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Table  3:  Summary  Statistics  for  DANA  Measures 


Measure 

Test 

Time 

n 

Min 

Qi 

Median 

Mean 

SD 

Q3 

Max 

1 

18 

757.20 

952.00 

1142.00 

1122.00 

202.83 

1274.00 

1432.00 

CSD 

2 

17 

818.30 

934.60 

1016.00 

1021.00 

128.79 

1082.00 

1279.00 

3 

15 

816.10 

911.00 

990.20 

1113.00 

313.96 

1185.00 

1888.00 

1 

18 

990.20 

1122.00 

1208.00 

1252.00 

208.81 

1360.00 

1637.00 

CSL 

2 

17 

897.90 

1001.00 

1104.00 

1121.00 

166.53 

1159.00 

1476.00 

3 

15 

906.40 

990.40 

1090.00 

1121.00 

184.84 

1195.00 

1612.00 

1 

18 

523.60 

571.50 

602.10 

617.60 

63.72 

642.70 

761.20 

Mean  RT 

PRT 

2 

17 

494.50 

544.60 

574.80 

577.90 

48.47 

604.90 

671.90 

3 

15 

476.40 

524.40 

580.90 

574.80 

65.68 

616.00 

718.70 

1 

18 

251.60 

281.70 

309.60 

308.20 

37.38 

325.60 

402.30 

2 

18 

267.80 

293.10 

304.70 

317.20 

37.79 

341.70 

423.50 

SRT 

3 

17 

251.10 

280.80 

303.10 

303.20 

36.84 

316.30 

390.40 

4 

17 

267.20 

305.40 

326.40 

323.90 

29.02 

344.90 

378.30 

5 

15 

275.80 

301.80 

316.50 

317.90 

28.44 

336.50 

383.00 

6 

15 

261.00 

306.20 

313.90 

313.60 

27.55 

329.70 

366.00 

1 

18 

37.05 

45.29 

50.20 

52.22 

10.39 

60.68 

75.33 

CSD 

2 

17 

40.95 

54.21 

56.76 

57.16 

8.67 

61.76 

73.32 

3 

15 

29.90 

50.00 

58.40 

54.91 

12.28 

62.24 

72.09 

1 

18 

36.45 

44.05 

48.77 

48.61 

7.74 

53.39 

60.59 

CSL 

2 

17 

40.68 

51.64 

53.74 

53.76 

7.41 

58.12 

65.94 

3 

15 

37.10 

48.72 

54.84 

53.84 

7.71 

60.10 

64.16 

1 

18 

81.36 

93.35 

98.40 

97.82 

8.95 

102.80 

114.60 

Mean  Thruput 

PRT 

2 

17 

86.74 

97.85 

104.60 

103.50 

7.96 

108.50 

117.00 

3 

15 

83.49 

96.22 

103.30 

104.80 

11.37 

112.60 

121.70 

1 

18 

149.20 

184.30 

196.20 

197.50 

22.63 

213.00 

238.50 

2 

18 

142.20 

175.60 

196.90 

191.50 

20.71 

204.70 

224.10 

SRT 

3 

17 

155.10 

189.70 

198.00 

200.80 

22.98 

213.70 

239.00 

4 

17 

158.60 

174.00 

183.80 

187.00 

17.65 

201.60 

224.50 

5 

15 

156.60 

178.40 

189.60 

190.70 

17.66 

198.80 

224.90 

6 

15 

164.00 

182.80 

191.10 

193.00 

17.74 

195.90 

229.90 
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Table  4:  Summary  Statistics  for  ANAM  Measures 


Measure 

Test 

Time 

n 

Min 

Qi 

Median 

Mean 

SD 

Q3 

Max 

1 

18 

745.00 

909.80 

1028.00 

1044.00 

221.79 

1154.00 

1546.00 

CSD 

2 

17 

791.00 

910.00 

1004.00 

1119.00 

304.24 

1167.00 

1907.00 

3 

15 

720.00 

848.00 

969.00 

1025.00 

265.25 

1154.00 

1729.00 

1 

18 

703.00 

834.20 

888.00 

936.60 

181.09 

1001.00 

1385.00 

CSL 

2 

17 

773.00 

855.00 

927.00 

965.90 

148.44 

1088.00 

1283.00 

3 

15 

749.00 

823.50 

860.00 

931.10 

178.00 

1014.00 

1415.00 

1 

18 

482.00 

517.50 

539.50 

545.10 

41.79 

566.20 

614.00 

Mean  RT 

PRT 

2 

17 

427.00 

506.00 

533.00 

536.60 

62.34 

577.00 

671.00 

3 

15 

486.00 

497.50 

532.00 

532.10 

40.04 

560.00 

613.00 

1 

18 

225.00 

234.00 

241.00 

252.90 

27.63 

267.20 

311.00 

2 

18 

222.00 

244.20 

256.50 

258.40 

23.84 

268.00 

311.00 

SRT 

3 

17 

215.00 

238.00 

246.00 

280.80 

122.33 

276.00 

744.00 

4 

17 

215.00 

237.00 

277.00 

301.00 

125.69 

293.00 

744.00 

5 

15 

125.00 

219.00 

228.00 

233.60 

41.34 

257.00 

314.00 

6 

15 

202.00 

240.00 

254.00 

259.20 

37.07 

275.00 

365.00 

1 

18 

18.00 

46.50 

55.50 

54.22 

15.90 

65.25 

78.00 

CSD 

2 

17 

20.00 

39.00 

57.00 

49.24 

14.16 

59.00 

63.00 

3 

15 

25.00 

49.00 

60.00 

57.27 

15.73 

65.50 

79.00 

1 

18 

43.00 

59.25 

66.00 

64.56 

10.37 

69.75 

83.00 

CSL 

2 

17 

46.00 

54.00 

62.00 

61.71 

8.70 

69.00 

75.00 

3 

15 

41.00 

58.00 

68.00 

65.13 

10.38 

71.00 

80.00 

1 

18 

97.00 

101.00 

107.00 

107.50 

7.33 

112.80 

120.00 

Mean  Thruput 

PRT 

2 

17 

85.00 

104.00 

111.00 

110.50 

12.73 

119.00 

137.00 

3 

15 

98.00 

105.00 

109.00 

110.00 

7.79 

116.00 

121.00 

1 

18 

193.00 

225.20 

246.00 

237.90 

22.89 

256.00 

264.00 

2 

18 

193.00 

224.00 

233.50 

234.00 

20.59 

245.80 

270.00 

SRT 

3 

17 

197.00 

226.00 

246.00 

240.90 

24.23 

252.00 

279.00 

4 

17 

135.00 

213.00 

227.00 

227.20 

35.77 

253.00 

280.00 

5 

15 

191.00 

233.50 

262.00 

251.80 

26.65 

269.00 

290.00 

6 

15 

164.00 

218.00 

236.00 

235.50 

30.05 

250.00 

297.00 
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Figure  4:  Paired  Differences  in  Mean  Reaction  Time  Within  Device 
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Figure  5:  Paired  Differences  in  Mean  Thruput  Within  Device 
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Table  5:  Wilcoxon  Signed  Rank  Tests  for  Differences  Between 
DANA- measured  and  ANAM-measured  Thruput  During  tests  of 
SRT  and  PRT 


Test 

Mo 

t 

w 

P-Val 

1 

84 

0.9661 

2 

71 

0.5509 

3 

73 

0.8900 

SRT 

-40 

4 

68 

0.7119 

5 

11 

0.0034 

6 

49 

0.5614 

1 

93 

0.7660 

PRT 

-10 

2 

94 

0.4307 

3 

84 

0.1876 
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Appendix  U 


MEMORANDUM 


To:  Cori  Lathan 
From:  Helaine  Resnick 
Re:  Altitude  Data 
Date  July  7,  2013 
Cc:  Lawrence  Wolpert 


This  memo  summarizes  findings  from  analysis  of  the  DANA  "altitude  data."  This  set  of  analyses 
addressed  a  number  of  questions  pertaining  to  an  experiment  in  which  DANA  was  used  at  different 
altitudes  to  assess  various  features  of  cognitive  function.  A  number  of  questions  related  to  specific 
variables  that  were  collected  in  this  experiment  are  listed  below  in  bold,  and  the  responses  to  each 
are  provided  immediately  below  the  question.  Some  questions  are  supported  by  supplemental 
graphics  that  illustrate  the  summary  of  findings.  A  number  of  the  questions  are  inter-related. 

There  was  interest  in  understanding  whether  using  mean  values  yielded  significantly  different 
results  from  use  of  median  values.  This  interest  was  based  in  concerns  about  the  appearance  of 
occasional  outlying  values  and  the  potential  impact  that  outliers  may  have  on  results.  Thus,  in  many 
analyses,  we  present  results  using  both  mean  and  median  SRT  values.  In  general,  the  conclusions 
that  are  drawn  are  similar,  although  it  is  clear  that  outliers  do  influence  the  distributional  properties 
of  the  data. 

The  data  set  that  was  used  in  these  analyses  contained  some  missing  data.  In  some  cases,  individual 
subjects  did  not  have  SRT  data  at  each  altitude;  in  other  cases,  participants  did  not  have  repeat 
administrations  of  SRT.  Because  part  of  these  analyses  involved  calculation  of  new  variables  based 
on  data  in  the  spreadsheets  that  were  provided,  there  are  some  instances  where  missing  data 
resulted  in  individual  participants  being  dropped  from  some  of  the  analyses.  These  missing 
observations  do  not  materially  impact  the  overall  findings  that  are  presented  below. 


1.  Does  DANA  SRT  detect  cognitive  deficit  measured  by  SRT  as  altitude  increases? 

Yes.  Anthrotronix  already  had  reason  to  believe  that  DANA  SRT  provides  useful  information  on 
cognitive  changes  with  varying  altitude.  Our  analyses  support  this  conclusion. 

The  chart  below  shows  box  and  whisker  plots  for  the  variable  "MeanCorrRT"  across  altitudes. 
These  show  quartiles,  minima  and  maxima.  Keep  in  mind  that  the  number  of  subjects 
contributing  data  is  quite  small  at  the  end  of  the  protocol  relative  to  the  beginning,  and  that  the 
overall  sample  is  relatively  modest  in  size.  The  left  panel  shows  means  for  the  first  SRT 
administration  (outliers  are  apparent),  and  the  right  panel  shows  means  for  the  second 
administration.  We  see  two  trends:  First,  SRT  becomes  significantly  less  favorable  on  the  first 
day  at  17,000  feet  (ALT1),  and  it  tends  to  go  back  toward  baseline  levels  by  day  16  at  high 
altitude.  The  second  finding  is  that  the  second  administration  of  the  SRT  protocol  (ADMIN2) 
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tends  to  have  less  favorable  values  at  each  altitude  relative  to  the  first  administration  (ADMIN1). 
This  finding  is  explored  further  below. 
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The  chart  below  shows  a  corresponding  set  of  analyses  using  median  SRT,  rather  than  mean 
SRT.  A  similar  pattern  is  observed  for  median  values,  both  with  regard  to  worsening 
performance  at  ALT1,  improvement  at  ALT16,  and  in  terms  of  less  favorable  performance  on  the 
second  administration  of  the  SRT  protocol  at  all  altitudes. 
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There  was  a  related  question  regarding  "retention  of  acclimation."  That  is,  do  participants  retain 
cognitive  function  after  having  come  down  to  sea  level  and  then  going  back  up  again.  These 
questions  can  be  explored  to  some  degree  with  data  that  were  collected  at  P7  and  P21,  the  time 
points  with  relatively  few  observations.  Nonetheless,  inspection  of  the  box  plots  above  indicates 
that  among  participants  for  whom  data  were  available,  performance  at  P7  was  actually  less 
favorable  than  ALT1  and  certainly  less  favorable  than  BL1.  Once  again,  ADMIN2  results  appear 
slightly  less  favorable  than  ADMIN1  results  at  P7. 


Does  a  second  administration  (trial)  of  the  DANA  SRT  protocol  provide  additional  useful 
information  on  cognitive  function  beyond  the  first  measure,  and  if  so,  how  does  this 
additional  information  "behave"  at  sea  level  (BL1),  at  the  first  day  at  high  altitude,  and  at  the 
16th  day  at  high  altitude  (acclimation)?  Are  the  differences  uniform  across  the  points  on  the 
protocol  timeline? 

The  second  SRT  administration  provides  useful  information.  The  box  plots  suggest  that  the 
overall  distribution  of  SRT  data  is  less  favorable  during  the  second  administration.  However,  it  is 
instructive  to  look  at  the  data  on  an  individual  basis  to  determine  if  this  pattern  is  consistent. 

We  can  do  this  by  creating  two  new  variables  "ADMIN2-ADMIN1"  for  both  means  and  medians. 
If  this  difference  is  generally  greater  than  zero  overall  and  for  each  individual,  we  may  conclude 
that  the  second  administration  yields  consistently  less  favorable  results  than  the  first.  The  chart 
below  shows  the  overall  distribution  of  the  two  difference  variables  (means  and  medians)  at 
each  altitude.  It  is  evident  that  the  overall  value  exceeds  zero  for  both  the  means  and  medians, 
with  some  variability  due  to  small  sample  size  and  outliers.  The  impact  of  the  outliers  (shown  as 
dots)  can  be  seen  for  the  difference  variables  in  much  the  same  way  as  they  could  for  the  earlier 
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analyses.  Once  again,  it  is  important  to  note  that  there  is  less  data  available  after  ALT16,  so 
those  results  should  be  interpreted  with  particular  caution.  Nonetheless,  P7  and  P21  difference 
variables  are  both  above  zero. 


ADMIN2-ADMIN1  for  MeanCorrRT  and  MedianCorrRT  by  Altitude 

O 

O  -  __ 

C\l 


1-BL1  2  -  ALT  1  3  -  ALT  1 6  4  -  P7  5-P21  6  -  BL2 


Difference  for  MeanCorrRT 


Difference  for  MedianCorrRT 


Another  way  to  look  at  the  data  is  to  examine  differences  on  the  individual  level  at  each 
altitude.  In  many  cases  this  is  not  practical  (or  advised)  because  such  anecdotal  findings  are 
often  misleading  and  rarely  robust.  In  this  case,  we  have  a  relatively  small  sample  size  as  well  as 
missing  data  at  various  points  in  the  protocol.  This  makes  it  easy  to  show  individual  level  data  on 
a  single  chart  with  the  understanding  that  we  are  not  engaged  in  inferential  statistics,  but 
rather,  we  are  looking  to  get  a  deeper  understanding  of  how  various  measures  behave  under 
different  experimental  circumstances. 

The  charts  below  show  these  individual-level  ADMIN2-ADMIN1  difference  data  for  both  means 
and  medians.  The  first  two  charts  look  at  these  differences  by  putting  BL1  and  ALT1  next  to  one 
another  on  the  same  graph.  For  participant  number  16  for  example,  the  mean  difference  in  SRT 
at  baseline  was  16  ms,  but  at  ALT1  it  was  57  ms.  In  most,  but  not  all  participants,  mean 
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differences  at  both  BL1  and  ALT1  were  greater  than  zero,  suggesting  poorer  performance  on  the 
second  administration.  Further,  in  many,  but  not  all  cases,  the  differences  at  ALT1  were  greater 
than  they  were  for  BL1,  suggesting  exacerbation  of  differences  under  the  experimental  stress  of 
altitude.  Analysis  of  median  values  showed  the  same  general  findings.  The  impact  of  the  outliers 
is  very  apparent  in  graphs  of  individual-level  data.  In  fact,  we  believe  that  these  are  the 
participants  that  Anthrotronix  personnel  indicated  would  not  be  included  in  routine  data 
analyses  because  of  their  suspect  measures. 


ADMIN2-ADMIN1  Difference  in  MeanCorrRT  BL1  vs  ALT1 
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ADMIN2-ADMIN1  Difference  in  MedianCorrRT  BL1  vs  ALT1 
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The  next  two  plots  show  individual-level  mean  and  median  differences  in  SRT  (ADMIN2- 
ADMIN1),  but  place  findings  for  ALT1  next  to  ALT16  to  illustrate  how  these  variables  behave  in 
relation  to  the  issue  of  acclimation  at  high  altitude.  We  see  the  same  pattern  indicating  that 
differences  are  generally  greater  than  zero  for  most  participants,  suggesting  poorer 
performance  on  the  second  SRT  administration.  But  when  we  look  at  ALT1  and  ALT16,  we  see 
that  for  differences  in  both  means  and  medians,  most  people  perform  much  worse  at  ALT1  than 
at  ALT16,  as  we  would  expect. 
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ADMIN2-ADMIN1  Difference  in  MeanCorrRT  AL1  vs  ALT16 
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ADMIN2-ADMIN1  Difference  in  MedianCorrRT  AL1  vs  ALT16 


54.5 


-84 


4  5  6  7  10  11  12  13  14  15  17  18  19  20  21  23  25 

I  diffmediancorrrt  altl  diffmediancorrrt  altl 6 


Overall,  the  results  of  ADMIN2  are  sufficiently  different  from  ADMIN1,  both  in  distribution  of 
the  data  and  in  systematically  higher  response  times  at  all  altitudes  to  suggest  that  ADMIN2 
provides  addition  information  beyond  that  which  is  offered  by  ADMIN1.  ADMIN2  also  had  a 
wider  range  of  Q1/Q3  (25th  and  75th  percentiles)  and  minimums  and  maximums  at  each 
altitude  compared  to  ADMIN1,  indicating  greater  spread. 

3.  Which  measure  of  reaction  time  is  the  most  sensitive  in  picking  up  declining  cognitive  function 
as  altitude  increases?  Is  it  ADMIN1,  ADMIN2  or  ADMIN2-ADMIN1? 

We  can  probably  conclude  that  ADMIN1  is  not  the  best  way  to  assess  reaction  time  because  we 
know  that  ADMIN2  is  generally  worse.  As  a  result,  the  difference  between  ADMIN2  and  ADMIN1 
is  going  to  be  greater  than  zero  in  most  cases.  Thus,  we  are  left  with  the  question  of  whether  to 
use  ADMIN2  or  ADMIN2-ADMIN1  to  assess  cognitive  decline.  For  the  sake  of  simplicity,  in  a 
research  setting  in  which  outliers  are  removed  from  the  data  set  with  certainty,  ADMIN2  is  likely 
to  provide  a  good  assessment  of  cognitive  decline  across  changing  altitude.  Further,  analysis  of 
the  difference  between  the  two  administrations  can  be  viewed  as  a  tool  that  helped  provide  a 
rationale  for  using  the  second  administration,  but  using  the  difference  itself  is  not  informative 
for  understanding  cognitive  function-we  need  to  use  a  variable  that  uses  the  full  range  of  SRT 
values,  not  just  a  plausible  range  of  differences.  One  way  to  think  about  this  problem  is  to 
compare  it  to  office-based  blood  pressure  measurements.  In  many  cases,  patients  are  anxious 
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on  their  first  measure,  making  it  artificially  elevated.  As  a  result,  a  second  measure  is  taken  and 
that  measure  if  often  used.  In  some  cases,  the  average  of  the  two  measures  is  preferred.  In  no 
cases  is  there  a  clinical  application  for  the  difference  in  two  blood  pressure  measures.  The  same 
principle  holds  true  for  the  second  SRT  administration.  If  use  of  both  ADMIN1  and  ADMIN2  is 
desired,  then  exploration  of  the  question  of  averaging  the  two  measures  is  more  useful  than 
using  the  difference  between  the  two. 

4.  Are  the  ADMIN1  and  ADMIN2  tests  similar,  or  does  ADMIN2  show  fatigability  (not  able  to 
sustain  the  response)?  If  non-zero  difference  are  observed,  is  this  fatigue  or  indifference  to 
performing  the  test  a  second  time? 

As  noted  above,  ADMIN2  shows  more  dispersion  and  higher  response  times  on  the  various 
charts  and  box  plots  for  both  means  and  medians  compared  to  ADMIN1.  The  higher  response 
times  could  indicate  participant  fatigue/difficulty  in  sustaining  high  performance,  or  indifference 
toward  the  second  administration.  It  is  not  possible  to  absolutely  distinguish  a  participant  who 
is  "annoyed"  with  having  to  do  the  protocol  twice  from  one  who  is  truly  fatigued.  However,  the 
consistency  of  the  patterns  with  regard  to  ADMIN2  vs  ADMIN1  suggest  that  participants  may 
indeed  be  unable  to  sustain  their  SRT  performance  levels  over  repeated  administrations  of  the 
testing.  Removal  of  outliers  from  the  data  set  would  likely  make  these  patterns  even  more 
homogenous,  thereby  supporting  the  idea  that  the  data  reflect  true  fatigue. 

5.  When  going  from  BL1  at  sea  level  to  ALT1  followed  by  ALT16  after  a  period  of  stabilization, 
how  much  acclamation  did  participants  retain  at  ALT16?  Is  there  an  impact  on  reaction  time 
by  the  end  of  the  experiment? 

ALT16  results  consistently  resembled  those  at  BL1,  suggesting  that  participants  acclimated  to 
altitude  by  day  16.  The  difference  in  MeanCorrRT  box  plot  medians  for  response  times  show 
that  they  decreased  (improved)  after  a  period  of  adjustment  at  that  altitude.  Response  times 
were  clearly  worse  on  the  first  day  at  17,000  ft  compared  to  BL1.  However,  by  ALT16,  response 
times  had  regressed  toward  BL1  values.  Along  the  same  lines,  P7  had  a  higher  median  response 
time  than  P21  compared  to  baseline.  However,  this  is  based  on  a  very  small  number  of 
observations  and  should  be  interpreted  with  extreme  caution. 
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MEMORANDUM 


To:  Cori  Lathan 
From:  Helaine  Resnick 
Re:  Altitude  Data 
Date  July  18,  2013 
Cc:  Lawrence  Wolpert 


This  memo  provides  additional  thoughts  on  several  questions  that  were  posed  in  response  to  our 
memo  dated  July  7,  2013.  Those  questions  are  in  bold,  followed  by  non-bolded  responses. 

Page  1:  Looking  at  the  data  -  medians  properly  classified  the  subjects  with  the  "bad"  data  better 
than  the  mean  data  but  frankly  those  subject  admins  would  never  stay  in  any  analysis  since  they 
had  so  few  trials.  If  they  can't  do  the  test,  they  are  either  malingering  or  sick  and  it  is  pretty  easy 
to  distinguish! 

Agreed,  the  median  did  a  better  job  of  identifying  outliers.  Since  this  was  a  qualitative  and  not  a 
statistical  analysis,  the  outliers  were  retained  for  the  box  plots  and  other  results.  Regardless,  the 
outliers  do  not  participate  in  the  calculation  of  the  percentiles  including  the  actual  median  values. 
Note  that  all  of  the  outliers  were  due  to  ID  15  and  ID  20. 

Page  3:  Are  you  saying  P7  is  actually  WORSE  than  ALT1? 

A  key  question  is  -  is  there  any  retention  of  acclimations  from  ALT16  to  P7.  You  are  saying  no. 

ALT1  had  a  lower  median  response  time  compared  to  P7  as  highlighted  below.  The  same  can  be 
seen  for  ALT16-those  values  are  lower  than  P7.  However,  P7  is  based  on  few  observations. 


Comparison  of  ALT1  to  P7  -  ALT1  has  a  lower  response  time  for  ADMIN1  & 
ADMIN2 


ADMIN 

Altitude 

Box  Plot  Medians  -  excludes 
outliers  from  the  calculation  of 
the  percentiles 
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Comparison  of  ALT16  to  P7  -  ALT16  has  a  lower  response  time  for  ADMIN1 
&  ADMIN2 
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outliers  from  the  calculation  of 

the  percentiles 
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Page  4:  Looking  at  individuals  and  "classifying"  them  is  exactly  what  you  do  with  hypoxic  subjects 
from  a  behavioral  perspective.  Not  sick,  sick,  very  sick.  If  we  could  predict  from  this  data  which 
ones  fell  in  to  which  category,  that  would  be  cool! 

Below  are  some  thoughts  about  classifying  the  individuals  into  categories  for  hypoxia.  These  ideas 
may  or  may  not  be  the  most  appropriate  approaches  to  address  the  question.  Further  evaluation 
can  be  done  to  identify  the  best  statistical  method(s)  to  meet  your  objective  if  you  wish  to  pursue 
this  idea  with  the  data  that  are  available. 


One  approach  would  be  to  use  percentiles  to 
classify  the  groups  by  using  a  combination  of 
percentiles  and  the  interquartile  range  (IQR).  A 
diagram  of  the  box  plot  with  the  corresponding 
information  for  the  distribution  of  data  is  provided 
for  reference.  The  IQR  corresponds  to  a  little  less 
than  +/-  one  standard  deviation.  Anything  above 
Q3  would  be  considered  either  sick  or  very  sick. 

Two  standard  deviations  could  be  used  as  the 
threshold  for  very  sick. 

Another  approach  is  to  perform  regression  analysis. 
However,  the  low  sample  size  may  be  a  problem. 
Since  regression  analysis  is  really  modeling  the  data  collected,  a  predictive  model  would  require  a 
second  dataset  to  confirm  the  model  or  the  data  would  have  to  be  split  into  two  datasets  to  develop 
and  validate  the  model. 

A  third  possibility  would  be  to  use  nonparametric  analysis  to  determine  the  category  of  not  sick,  sick 
or  very  sick.  This  might  be  a  better  approach  due  to  the  small  sample  size. 

Page  8:  I  don't  know  why  you  would  say  this  -  the  difference  may  be  much  more  reflective  of 
cognitive  function  than  the  actual  value.  The  actual  value  doesn't  tell  you  anything  without  a 
comparison  either  (eg.  the  difference  from  baseline). 

We  were  not  referring  to  differences  from  baseline,  but  differences  at  a  single  point  in  time 
between  two  separate  administrations.  The  statement  on  page  8  was  based  on  comparing  the 
information  on  MeanCorrRT  for  ADMIN1  and  ADMIN2  separately  versus  looking  at  the  difference 
between  ADMIN2-ADMIN1.  While  both  methods  use  the  same  data,  the  first  method  allows  the 
individual  reviewing  the  results  to  visually  compare  ADMIN  1  versus  ADMIN2  and  determine  if  the 
results  are  similar  or  different  for  the  response  time  and  the  distribution  of  response  times  for  each 
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ADMIN.  ADMIN2-ADMIN  gives  a  direct  measure  of  the  difference  in  response  time  but  may  be 
more  difficult  to  look  at  differences  in  patterns  between  ADMIN1  and  ADMIN2  response  times. 
Essentially,  it  may  be  more  of  a  preference  in  how  one  thinks  about  data  and  what  is  needed  to 
address  the  question  of  cognitive  function. 

Page  8:  Totally  not  valid  analogy!  A  much  better  one,  although  not  perfect  is  a  thermometer.  You 
never  take  it  a  2nd  time.  The  measurement  is  expected  to  be  valid  on  first  try  unless  some  strange 
circumstance.  SRT  is  much  more  like  that  -  a  reflex  -  either  you  are  malingering  or  sick!  There  is 
no/little  learning  or  relaxing.  In  fact,  a  little  stress/adrenaline  would  be  more  likely  to  make  the 
reaction  time  faster,  not  slower,  thereby  decreasing  the  diff  b/n  srtl  and  srt2. 

Agreed  that  your  analogy  is  better-we  will  stick  to  statistics! 

Page  9:  This  is  true  ONLY  for  baseline.  Actually  what  is  recommended  for  neurocog  baseline  is 
take  it  3  times  and  take  BEST  score 

Ok. 

Page  9:  Outliers  -  are  you  referring  to  the  3  ALT1  admins  that  should  be  deleted? 

The  calculation  of  the  box  plot  percentiles,  including  the  median,  do  not  include  the  outliers.  The 
three  "outliers"  you  mention  are  the  green  lined  data  in  the  SRT  spreadsheet  provided  for  analysis? 
If  so,  the  "outlier"  for  MeanCorrRT=480  for  ALT1,  ADMIN2,  ID  12  is  not  an  outlier  on  any  of  the  box 
plots  presented  on  pages  2-4. 

Page  9:  The  "retention"  question  is  for  P7  and  P21.  After  16  days  of  acclimation,  they  go  back 
down  to  about  5,000  for  7  or  21  days.  P7  is  back  at  altitude  after  7  days  and  the  question  is  are 
they  more  like  ALT1  or  ALT16. 1  think  they  are  more  like  ALT16  -  is  that  true? 


For  both  ADMIN1  and  ADMIN2,  the  MedianCorrRT  box  plot  median  for  P7  is  more  similar  to  ALT1 
than  ALT16. 

For  both  ADMIN1  and  ADMIN2,  the  MedianCorrRT  box  plot  median  for  P21  is  more  similar  to  ALT16 
than  ALT1. 
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SUMMARY 

Collection  of  data  from  50  normal  participants  (no  head  injury,  currently  healthy,  non- 
deployed)  in  each  of  several  age  categories  will  provide  sufficient  information  to  define 
thresholds  of  normal  values  for  DANA  subscales.  Our  pilot  data  not  only  suggest  that  50 
participants  is  a  conservative  number,  but  they  also  indicate  that  this  number  is 
sufficient  to  provide  a  favorable  trade-off  among  considerations  as  diverse  as  project 
operation  costs,  clinical  sensitivity  and  potential  future  applications  of  DANA  as  a 
screening  and  triage  instrument. 

BACKGROUND  AND  STATEMENT  OF  FACT 

We  were  asked  to  respond  to  concerns  regarding  the  absence  of  a  "power  analysis" 
underpinning  our  recommendation  to  enroll  50  individuals  per  age  group  as  part  of  a 
larger  effort  to  establish  a  normative  database  of  DANA  values.  Two  points  should  be 
recognized: 

1 .  A  power  analysis  is  one  of  a  number  of  standard  practices  that  are  used  to  design 
clinical  studies  where  the  goal  is  to  engage  in  hypothesis  testing.  In  many  cases,  such  a 
hypothesis  aims  to  determine  if  two  groups  are  statistically  distinct  from  one  another 
(e.g.  a  randomized  clinical  trial  to  test  the  effectiveness  of  a  new  therapy  relative  to 
standard  of  care).  Normative  databases  are  not  used  to  compare  groups  in  this  way.  As 
a  result,  classic  power  analyses  do  not  apply  in  this  setting. 

2.  There  is  no  "standard  number"  of  individuals  that  is  universally  required, 
recommended  or  recognized  for  establishment  of  a  normative  database.  Nonetheless,  it 
is  recognized  that  "more  information  is  better"  until  the  point  of  diminishing  returns  is 
reached  with  regard  to  precision.  Identifying  the  point  at  which  this  condition  is  met  is  a 
subjective  process  involving  assessment  of  all  available  evidence  combined  with  clinical 
judgment. 

RATIONALE  FOR  RECOMMENDATION  OF  50  INDIVIDUALS  PER  AGE  GROUP 
The  attached  spreadsheet,  which  is  based  on  actual  DANA  reaction  time  data, 
illustrates  how  we  arrived  at  a  recommendation  of  50  people  per  group.  The 
spreadsheet  is  divided  into  five  age  groups  in  a  manner  that  reflects  how  we  anticipate 
enrolling  participants  into  the  normative  database  study.  The  reaction  time  data 
(mean=306.75  [column  A],  s.d.=46.56  [column  Bj)  for  the  youngest  age  group  (age  <30) 
is  based  on  pilot  data  collected  at  LeJeune  and  29  Palms.  Column  C  of  the  spreadsheet 
examines  scenarios  in  which  the  enrollment  for  a  hypothetical  normative  database 
varies  from  a  minimum  size  of  10  individuals  to  a  maximum  of  100  individuals  in  each 
age  group.  Using  the  standard  deviation  and  varying  sample  sizes,  we  calculate  the 
standard  error  for  each  scenario,  a  value  that  appears  in  column  D.  It  is  evident  that  the 
standard  errors  of  these  normative  database  scenarios  decrease  in  an  expected  pattern 
as  sample  size  increases. 


The  goal  of  a  normative  database  is  to  identify  threshold  values  at  which  individuals  are 
no  longer  considered  to  be  within  a  normal,  or  expected  range  of  values.  By  design,  the 
pilot  data  on  which  our  sample  size  estimates  are  based  consist  entirely  of  young, 
normal  individuals,  and  are  therefore  tightly  distributed.  Against  the  backdrop  of  these 
highly  informative  pilot  data,  we  must  determine  at  what  threshold  we  wish  to  define  an 
individual  as  being  outside  the  normal  range  in  the  future.  Because  all  DANA  measures 
are  unidirectional  with  respect  to  performance  (e.g.  "higher  is  better"  or  "lower  is 
better"),  we  are  only  concerned  with  upper  or  lower  thresholds  of  normal,  but  not  both. 

In  the  reaction  time  example,  we  are  concerned  with  identifying  an  upper  threshold  of 
normal  because  shorter  reaction  times  indicate  better  performance.  Although  the 
converse  is  true  for  other  DANA  measures,  the  following  approach  applies  equally  to 
both  measurement  settings. 

A  rigorous  strategy  to  identify  and  evaluate  upper  thresholds  of  normal  is  to  calculate 
the  value  of  reaction  times  or  other  DANA  parameters  that  correspond  to  critical 
locations  on  the  upper  end  (right  hand  side)  of  the  reaction  time  distribution.  These 
locations  correspond  to  critical  values  of  the  t  distribution  and  are  shown  in  Columns  E 
through  H  of  the  spreadsheet.  For  example,  10%  of  data  on  the  t  distribution  fall 
beyond  (to  the  right  of)  1.372  and  0.5%  of  the  data  fall  beyond  3.169.  We  can  use 
these  critical  t  values  to  calculate  actual  DANA  values  such  that  we  can  identify 
thresholds  beyond  which  10%,  5%,  1%  or  only  0.5%  of  the  data  are  likely  to  fall.  We  can 
then  label  individuals  who  fall  in  these  parts  of  the  curve  "abnormal"  depending  on  how 
conservative  we  wish  to  be  with  this  definition.  Because  these  threshold  calculations 
involve  the  standard  error  (which  is  based  on  sample  size),  they  will  change  in  response 
to  changing  sample  sizes.  Thresholds  of  interest  are  shown  in  columns  J  through  M  of 
the  spreadsheet,  and  are  the  basis  of  our  recommendation  to  enroll  50  individuals  per 
age  group. 

It  is  evident  that  in  scenarios  using  smaller  sample  sizes,  the  likelihood  of  a  false 
negative  result  increases.  For  example,  if  only  10  people  in  the  <30  age  group  were 
enrolled  in  the  normative  database  study,  99.5%  of  the  data  in  this  group  would  be 
expected  to  fall  below  a  threshold  of  353  ms,  and  these  individuals  would  be  considered 
normal.  In  contrast,  a  normative  database  that  includes  50  individuals  would  place  that 
same  threshold  at  324  ms,  a  level  that  will  result  in  a  larger  number  of  individuals  falling 
beyond  the  normal  threshold  value.  This  will  reduce  the  frequency  with  which  the 
normative  database  will  result  in  false  negatives.  Another  way  to  express  this  concept 
is  to  say  that  the  database's  sensitivity  will  increase.  Clinically,  increasing  sample  size 
for  a  normative  database  will  result  in  DANA  being  able  to  distinguish  between  normal 
and  abnormal  individuals  with  greater  precision.  The  advantages  of  such  a  result  are 
obvious. 

Examination  of  threshold  values  across  sample  size  scenarios  indicates  that  regardless 
of  what  threshold  of  normal  (e.g.  >90%  or  >99.5%)  is  of  interest,  enrollment  of  additional 
subjects  yields  diminishing  returns  on  precision  after  approximately  50  individuals.  That 
is,  the  relative  contribution  of  participants  to  the  precision  of  the  database  is  most 
evident  at  sample  sizes  up  to  50,  at  which  point  the  calculated  thresholds  change  very 


little.  As  emphasized  above,  however,  sample  sizes  smaller  than  50  will  result  in  a 
potentially  unacceptable  number  of  false  negatives  and  low  sensitivity,  a  property  of  the 
database  that  will  have  obvious  clinical  and  tactical  implications. 

DANA  pilot  data  did  not  include  sufficient  numbers  of  older  service  members  to  yield 
usable  information  for  calculations  similar  to  those  in  the  <30  age  group.  As  a  result, 
threshold  values  for  the  older  groups  were  projected  based  on  application  of  two 
assumptions  to  the  data  for  the  <30  age  group.  Using  reaction  time  as  an  example,  we 
assumed  that  the  mean  value  would  increase  (i.e.  performance  would  become  less 
favorable)  by  10%  with  each  increasing  age  group.  We  also  assumed  that  the  standard 
deviation  would  increase  (i.e.  the  distribution  would  become  more  spread  out)  by  10% 
with  each  increasing  age  group.  Using  these  assumptions,  which  we  believe  are 
extremely  conservative,  we  nonetheless  see  that  enrollment  of  50  individuals  in  the 
older  age  groups  results  in  appropriate  levels  of  precision  with  respect  to  estimated 
threshold  values  and  that  enrollment  of  additional  individuals  exceeds  the  point  of 
diminishing  returns. 

CONCLUSION 

DANA  pilot  data  suggest  that  enrollment  of  50  normal  (no  head  injury,  currently  healthy, 
non-deployed)  individuals  per  age  group  into  the  normative  database  study  will  yield 
sufficient  data  to  calculate  age-specific  thresholds  for  DANA  values  that  can  be  used  to 
distinguish  normal  from  abnormal  individuals.  Based  on  the  data  that  are  currently 
available  to  us,  we  believe  that  enrollment  of  fewer  than  50  individuals  will  result  in 
unacceptably  high  numbers  of  false  negatives  and  low  sensitivity,  and  enrollment  of 
more  than  50  people  will  not  add  appreciably  to  the  precision  of  the  database  with 
regard  to  its  utility  as  a  screening  and  triage  instrument. 


Means  and  standard  deviations  are  based  on  10%  inflation  from  the  <30  age  group.  Any  numbers  can  be  inserted  for  the 
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Transitioning  the  Defense  Automated  Neurobehavioral  Assessment  (DANA) 

Tool  to  Operational  Use  (DANA-RIF)  Appendix  W 

Award  Number  W81XWH-12-C-0204 

PI:  Dr.  Corinna  Lathan  Org:  AnthroTronix,  Inc.  Award  Amount:  $3M 


Study/Product  Aim(s) 

•  Scientifically  and  clinically  validate  DANA 

•  Obtain  FDA  clearance  for  DANA 

•  Transition  DANA  to  DoD 


Approach 

Three  studies  were  conducted  to  validate  DANA’s  efficacy  in 
assessing  change  in  cognitive  function  as  a  result  of 
concussion,  depression  and  PTSD.  A  51 3(g)  determination 
from  the  FDA  will  decide  whether  a  510(k)  submission  is 
necessary,  and  DANA  will  be  transitioned  to  the  DoD.  We 
submitted  a  510(k)  and  DANA  received  FDA  clearance  on 
October  15,  2014. 


Timeline  and  Cost 


DANA  Brief  [NO  S] 

Administered  :°7/;9/,2012at  17:45 
by  Test 

Given  to:  Subject  1 

Overall  Results 

Details  I 

Results  for  Psychological 


Measures 

Test  Results 

PHQ-I 

PCPTSD 

ISI 

8 

Survey 

|  Survey  | 

Normative  Results  For 

r tnrniti\/p  TpsK 


Accomplishment:  All  three  studies  have  been  completed,  preliminary  data  analysis  has 
been  done,  full  data  analysis  is  on-going  and  manuscript  preparation  is  in-progress. 


Goals/Milestones  (Example) 

CY13  Goal  -  Obtain  FDA  clearance,  and  IRB  approval  for  three 


Activities  CY 

13 

14 

IRB  approval  for  scientific  studies 

Conduct  scientific  studies 

Obtain  FDA  clearance 

Transition  DANA 

Estimated  Budget  ($K) 

$1,800,000 

$1,200,000 

studies. 

/  Submitted  513(g)  RFI 
/  Submitted  51 0(k) 

/  Obtained  IRB  approval  for  all  three  studies. 

CY14  Goal  -  Conduct  scientific  studies,  publish  results,  transition 
DANA 

/All  studies  completed  data  collection. 

/  Numerous  manuscripts  have  been  published  showing  that  DANA  is  a 
reliable,  consistent  assessment  tool. 
Comments/Challenges/Issues/Concerns 
•  Delay  in  data  analysis  will  require  additional  publications  to  be 
completed  this  coming  year. 

Budget  Expenditure  to  Date 
Projected  Expenditure:  $3,000,000 
Actual  Expenditure:  $3,000,000 


Updated:  (September  9,  2015) 
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DEPARTMENT  OF  HEALTH  &  HUMAN  SERVICES 


Public  Health  Service 


Food  and  Drug  Administration 
10903  New  Hampshire  Avenue 
Document  Control  Center  -  WO66-G609 
Silver  Spring,  MD  20993-0002 


October  15, 2014 


AnthroTronix,  Inc. 

Dr.  Corinna  Lathan 
Chief  Executive  Officer 
8737  Colesville  Road,  Suite  L-203 
Silver  Spring,  MD  20910 

Re:  K141865 

Trade/Device  Name:  DANA 
Regulation  Number:  Unclassified 

Device  Classification  Name:  Recorder,  Attention  Task  Performance 

Regulatory  Class:  Unclassified 

Product  Code:  LQD 

Dated:  July  9,  2014 

Received:  July  10,2014 

Dear  Dr.  Lathan: 

We  have  reviewed  your  Section  510(k)  premarket  notification  of  intent  to  market  the  device 
referenced  above  and  have  determined  the  device  is  substantially  equivalent  (for  the  indications 
for  use  stated  in  the  enclosure)  to  legally  marketed  predicate  devices  marketed  in  interstate 
commerce  prior  to  May  28,  1976,  the  enactment  date  of  the  Medical  Device  Amendments,  or  to 
devices  that  have  been  reclassified  in  accordance  with  the  provisions  of  the  Federal  Food,  Drug, 
and  Cosmetic  Act  (Act)  that  do  not  require  approval  of  a  premarket  approval  application  (PMA). 
You  may,  therefore,  market  the  device,  subject  to  the  general  controls  provisions  of  the  Act. 

The  general  controls  provisions  of  the  Act  include  requirements  for  annual  registration,  listing  of 
devices,  good  manufacturing  practice,  labeling,  and  prohibitions  against  misbranding  and 
adulteration.  Please  note:  CDRH  does  not  evaluate  information  related  to  contract  liability 
warranties.  We  remind  you  however,  that  device  labeling  must  be  truthful  and  not  misleading. 

If  your  device  is  classified  (see  above)  into  either  class  II  (Special  Controls)  or  class  III  (PMA), 
it  may  be  subject  to  additional  controls.  Existing  major  regulations  affecting  your  device  can  be 
found  in  the  Code  of  Federal  Regulations,  Title  21,  Parts  800  to  898.  In  addition,  FDA  may 
publish  further  announcements  concerning  your  device  in  the  Federal  Register. 

Please  be  advised  that  FDA’s  issuance  of  a  substantial  equivalence  determination  does  not  mean 
that  FDA  has  made  a  determination  that  your  device  complies  with  other  requirements  of  the  Act 
or  any  Federal  statutes  and  regulations  administered  by  other  Federal  agencies.  You  must 
comply  with  all  the  Act’s  requirements,  including,  but  not  limited  to:  registration  and  listing  (21 
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CFR  Part  807);  labeling  (21  CFR  Part  801);  medical  device  reporting  (reporting  of  medical 
device-related  adverse  events)  (21  CFR  803);  good  manufacturing  practice  requirements  as  set 
forth  in  the  quality  systems  (QS)  regulation  (21  CFR  Part  820);  and  if  applicable,  the  electronic 
product  radiation  control  provisions  (Sections  531-542  of  the  Act);  21  CFR  1000-1050. 

If  you  desire  specific  advice  for  your  device  on  our  labeling  regulation  (21  CFR  Part  801),  please 
contact  the  Division  of  Small  Manufacturers,  International  and  Consumer  Assistance  at  its  toll- 
free  number  (800)  638-2041  or  (301)  796-7100  or  at  its  Internet  address 
http://www.fda.gov/MedicalDevices/ResourcesforYou/Industry/default.htm.  Also,  please  note 
the  regulation  entitled,  "Misbranding  by  reference  to  premarket  notification"  (21  CFR  Part 
807.97).  For  questions  regarding  the  reporting  of  adverse  events  under  the  MDR  regulation  (21 
CFR  Part  803),  please  go  to 

http://www.fda.gov/MedicalDevices/Safetv/ReportaProblem/default.htm  for  the  CDRH’s  Office 
of  Surveillance  and  Biometrics/Division  of  Postmarket  Surveillance. 

You  may  obtain  other  general  information  on  your  responsibilities  under  the  Act  from  the 
Division  of  Industry  and  Consumer  Education  at  its  toll-free  number  (800)  638-2041  or  (301) 
796-7100  or  at  its  Internet  address 

http://www.fda.gov/MedicalDevices/ResourcesforYou/Industry/default.htm. 


Sincerely  yours, 


Felipe  Aguel  -S 

,  Carlos  L.  Pena,  PhD,  MS 
f0r  Director 

Division  of  Neurological 

and  Physical  Medicine  Devices 
Office  of  Device  Evaluation 
Center  for  Devices  and  Radiological  Health 


Enclosure 


